DEV Community: Ray

The Spring Workload and Auditing My Progress

Ray — Wed, 12 May 2021 22:18:42 +0000

Where Ya Been

I planned to write a blog post at the end of February talking about how much I got done in the month - reading that post I laid out some pretty hefty goals for a single month. Especially while I was still working.

However, at the time I was working maybe 16 hours a week? At the very least it was an amount that made me feel as if I could have multiple days off a week to work on projects.

That didn't exactly go as planned. As one might be able to tell.

The main purpose of this blog is to audit the amount of work I got done over the last 3 and a half months and plot a course for the next couple. It's not meant to be a big long document where I kick myself for not working hard enough because frankly, that's unhealthy. Plus I have good reasons for not getting any work done!

These last few months have been... well not stressful, but busy. Work picked up in a way that I did not expect as inventory and the spring seasonal came in full force over March and April. I was working between 30 and 40 hours a week at that time and I don't blame myself for not having worked on projects. Likewise, for the last half of April I tested positive for covid-19 which is not an illness that is easy to work on the computer with, I'll tell you.

So fine, I don't think kicking myself for not getting that lengthy list of projects done is helpful so I won't do it. However, like I said previously, I would like to take a look at that list and plan how I will get it finished soon and move on to the many other ideas in my head.

Goals 2

1. Implement my second feature into RayAi

This was not done for a reason I'll reveal in a bit, but as well, I didn't work on it because I haven't felt terribly inspired to work on it. I think at least 1 more feature is something I want to get done pronto but after that I think I'll put this one on the back burner for some new projects I have in mind. It's already on the back burner for...

2. Move my personal website to a better url

This one was simple in hindsight but annoying the way I did it. So I was recommended to a friend to use a VPS provided by a cheap and efficient provider. I thought this was a great idea at first, but I quickly realized setting it up was much more annoying than a slightly more expensive provider that made things easier. I fooled around with the VPS for a while and after deciding that I didn't want to be a server administrator and instead wanted to be a developer, I switched to a much simpler provider that allowed me to deploy my static site at https://raymondrambo.ca/ overnight.

So that's a win!

3. Start on a React project

This one actually got done too! A family member has a small business and I offered to build them an e-commerce website from the ground up. I've run into a snag or two from learning more about React and I'm at the point in my development where I'm realizing I need to integrate the e-commerce elements from the start. But hopefully, I can finish this project up, since it feels like it's moving quickly, and move onto another React project.

So what's next?

Well, I don't think I'll be making a big showing of it in this blog post, but I have a few goals that I would like to accomplish before I write another.

The first is to finish up this e-commerce platform and put it under my belt. Having a project like this I feel will look really good and give a huge sense of satisfaction. Plus I can commission friends who have small businesses to build them something similar once I know how to do everything. :)

The second is to finally get to work on that Discord bot again. I like developing this bot for Discord but it takes a lot of time and I don't usually have the patience for it. It's nice the see the fruits of your labour though. I have ideas.

Finally, I would like to get started on a NEW new project, probably in React, and as well as the Discord bot, I have some ideas.

So these goals are in chronological order and if I get through the e-commerce platform and the Discord bot but don't start a new project, I'll be ok. But I absolutely would like to see these to fruition quickly.

Until next time!

February Start to Finish

Ray — Thu, 04 Feb 2021 20:00:43 +0000

Housekeeping

At the beginning of January, I wrote The first After-School Project that focused on how I would build my personal website. I learned a lot about simple, static, website designs and I still intend to keep working on that site and learn more about web hosting so I can move off GitHub pages, at least for my newer projects that aren't static. However, I'd like to talk about something besides that - namely, my newest project, and what I plan to have finished by the end of February.

We can easily ascertain the goal of most fledgling developers because it tends to be the goal of almost everyone in my position: find fulfilling, well-paying work in my field of choice. Namely: web development. To that end, I need to build up a portfolio of projects that I can show off as proof of my strong, unending deluge of programming skills.

Ok ok ok, that's enough housekeeping, let's talk about my project.

The Project

So this is a Discord bot. It's not terribly removed from what I've worked on before since I also worked a little on MHTimerBot adding a new feature and creating tests, as well as simple bug fixes.

Currently, the bot has 1 2-in-1 feature.

Right now, you can type ai nyaa and then a search keyword, and the bot will scour nyaa.si looking for hits that match your keyword. Then, you can react to the post and get a link to the download of the hit you're looking for.

You can take a look at the project here:

fluentinstroll / RayAi

Ray's Discord bot, meant to do whatever he finds to be needed - built in an entirely selfish way.

RayAi

Personal Discord Bot

RayAi is being built to facilitate whatever random things I want to have access to in a discord bot.

Features included:

nyaa.si lookup:

type ai nyaa <search string> to get the first 10 results of the nyaa.si search
react to the search to get a torrent link to the cooresponding item

View on GitHub

Goals

What I'd like to do now that I've introduced my new project is lay out some goals that I'd like to achieve by the end of February.

1. Implement my second feature into RayAi

Right now I have 1 real feature and I'd like to bump that up to 2 by the end of the month. My plan for the next feature is Monster Hunter related, I'd like to allow users to look up a monster or hunt, or something of the like and see possible drops or rewards. There can be a lot here and perhaps I can expand a command like this to multiple features but for now I'd like to stick to lookup hunt/monster to see rewards/drops.

2. Move my personal website to a better url

So for now I'm using fluentinstroll.github.io but that doesn't look great on an Indeed profile or a resume. To that end, I purchased a domain that better represents me and I plan to move my personal website to that.

I don't really know how to do that yet but at some point this month and look into it and quickly get it done.

3. Start on a React project

The next thing I want to do is learn more about how Reactjs works and how to build React apps. I have a couple of ideas for this but I'll keep them to myself for now. The goal for this is not to finish something substantial but to start something substantial that will have potential and will show off my abilities well.

Even if I get into March with a started project but not a full realized one, I'll consider this a success but I still intend on doing as much work on it as I can and hopefully can have a 0.1.0 release finished in time for March.

4. Refactor the RayAi code

Right now RayAi is a learning experience that I've learned a lot from but it's not necessarily a great example of stellar code. I think I need to refactor my current code, maybe even before working on the new feature, so that I can outline good practices for going forward.

A 4.1 for this might be to add a linter and code formatter to help keep the code uniform as well.

Conclusion

That's all I have to say for this blog, I think I laid out a decent number of attainable objectives here and I feel that I really can get this finished. Look forward to my end of February/ beginning of March blog post where I decide if I succeeded or failed in my quest to do literally any work.

The first After-School Project

Ray — Thu, 07 Jan 2021 04:44:17 +0000

Finishing OSD600 was a major milestone in my development as a programmer and as a software maintainer. I genuinely cannot express how happy taking that class has made me and how ready I feel to take on the next big project.

But first...

I want to make something simple. I took a break from coding for about a week after my class ended (and thus my stint at Seneca College in general), and decided to enjoy the holidays with the many hours I was given at my part-time job and my girlfriend, with whom I spent lots of time cooking food with on Christmas Day.

But back to the project. I want it to be simple and brush me back up on coding for the front-end. Though the projects I was working on in OSD600 were fun and great, I loved creating link checker and releasing it as well as working on other Javascript-heavy projects, I do feel like my back-end skills have been worked hard for now and though I will return to that, I want to learn more about webpage design and front-end frameworks like React and Vue (having worked largely with Angular the last year).

However, this first project won't be in either of those - it will be with Jekyll.

Yes, that's right, I building a personal website for myself. The project every software developer ruins into at least once. Allow me to outline what I want to build and what I would like to learn:

I would like to learn how to design appealing-looking web pages

The biggest thing for me is that as a software developer, I don't have a terribly great eye for beauty. That doesn't mean I don't understand the basics of how to make something pleasing to look at, it just means that I don't the skill required to design something that's more complicated than, say, a simple blog or table of information. I also am not that confident in my CSS knowledge and I'd like to fix that.

I would like to learn Jekyll

I was originally going to build my website in vanilla Javascript, but I think now that was foolish and instead I'd like to learn a static site generator that will work for simple webpages because even though I love Javascript, I don't want to work for a simple website.

I would like to learn about GitHub Pages

Frankly, I think being able to host my projects straight from GitHub would be such a great option since all my code is going to be there anyway. I know locking yourself into an ecosystem is kind of a bad idea but I don't see anyone raising alarms about it so I'm not too perturbed.

Let's start

First, I created my GitHub pages repository and connected it to a local git repo, then I started getting into Jekyll. I recommend this blog post if, like me, you've gotten used to using WSL in vscode for your web development work.

Jekyll

Getting started with Jekyll is pretty easy. GitHub Pages even has a section on initializing the Jekyll site with Pages.

Not that I have a templated site on my Pages domain, I'd like to make this blog, my dev.to blog, display on my site.

A great GitHub repo for this is equalcoding.github.io, with this template I managed to build a website in about 30 minutes that would display all my dev.to blogs and my GitHub repos.

The next things I would like to do are change up the look of the site and maybe add repos that I've recently contributed to? Or are in general involved with.

Though I didn't particularly learn a ton about Jekyll, since the site I cloned used a lot of vanilla Javascript and HTML, I think themes are very important and I can't wait to explore them more.

If you;d like to take a look at my site, you can see it here.

Until next time!

Release 0.4 and the End

Ray — Sat, 12 Dec 2020 04:50:40 +0000

I wanted to write this blog post on a triumphant, feel-good haze of power from smashing my biggest hurdle yet. However, I'm actually pretty behind on my open-source classwork.

I'll admit I started a little late, as my mid-point blog post laments, and the last night of work has been stressful, but I think I won't mind too much.

Not to say I won't miss the marks - but I'd be remiss if I didn't admit that the real reason I took this class and stuck with it was because it was a new experience that let me learn so many new things that I can apply to life as a developer.

I encourage you to read that mid-point blog post as well because it highlights everything I did while working on this pull request with a couple of exceptions I'll discuss here.

Building

My biggest fear was also my most correct prediction: working within other developers' codebases would be the most difficult part of this endeavor. I think figuring out that getMice leads to getSearchedEntity which then sends the info back to sendInteractiveSearchResult which then uses formatMice to build a table which then finally prints the message out for the Discord bot to say was a very difficult line to follow and I think it really highlights how starting early on something like this can help. I think that if I had started even a few days earlier, I would've had this line figured out and could've plotted my own with my own functions much sooner and with greater confidence.

The other difficult part of writing this feature was the testing. Frankly, working with testing frameworks has been the bane of my existence. I finally got used to Jest and now there's a whole other one I have to learn! Truly exhausting.

But seriously, the most difficult part of working with another developer's code is that it's their code. And it really makes the most sense just to them.

So if you remember from my last post, I described where I needed to go with my code.

The process is similar to the code in the other commands, it will include a way to search the agiletravels database for the correct item, then print a list of items that can be found in it for the user to peruse. Seems pretty easy right?

This is basically what I did, but following the lead with other code blocks, I created my own line through the program to get me to print out my own table of items. It turned out to be not that hard and I almost feel proud that I managed to get it working with relative ease.

I started this post by lamenting that I wanted to feel triumphant about my work, but to be quite honest, I am. I think that creating a feature like this to add to a program that with sit in dozens of discord servers and maybe be used by hundreds of people - I think it's a really great thing and I'm really happy to have developed a feature like this. If you want to take a look at the pull request and see how I'm doing you can look here.

I think the biggest thing that I missed out on was leveraging the open-source community, especially the one in the OSD600 slack channel. Everyone there is quite adept and is always working on stuff. Though I never really felt like I need to ask any questions, I think doing so would've helped me out in a lot of places.

Lessons

So before I close out this blog post, I wanted to talk a bit about what I learned. Not just that other peoples' code is hard to read. But more about what I learned about collaborating and the technical aspects of open source development.

One thing I learned is everyone on GitHub is always trying to be polite but sometimes just can't help themselves.

Also that while git just loves to screw up your line endings, it's really the repo owner's fault if they don't have a code formatter that runs before each commit isn't it?

Or at least, if git does go ahead and ruin your line endings, you can at least roll back and force push the good version.

Actually, nevermind that one.

I think as I get better as a developer I'll have a little more confidence and be a little more open about asking for help from others but for now, the biggest learning experiences have come from learning how to work the robust architecture of git and using ti to work with so many different people and I can't wait to apply it to my next project.

Release 0.4, the Halfway Point

Ray — Mon, 07 Dec 2020 23:43:53 +0000

Phew! It's been kind of a wild weekend, and a wild previous week. Unfortunately, this blog post was meant to be written at the end of last week, not the beginning of this week. However, life and work have their ways of messing with our plans.

I decided to take on a feature request on the Mousehunt Timer Bot and have been plugging away at it.

The process is similar to the code in the other commands, it will include a way to search the agiletravels database for the correct item, then print a list of items that can be found in it for the user to peruse. Seems pretty easy right?

The first step will be to take a look at other commands' code and think about how I can write a command that looks and acts like them. Then I need to look into how the bot displays the information (which I have done when I fixed a bug previously) and set up the correct message to display.

This will be another living document that I'll update as I work on this issue. Please stay tuned!

Session One

In my first work session, I started by laying out the process for creating this command.

Research the other commands and figure out how the internals of the discord bot really work - get to know the code.
Set up my command with copy-pasted code from a different command to show that I can at least get the bot to spit out information from my new file.
Write the actual command, taking code structure and design cues from the code of the other commands.
Bug test.
Write tests for my command and perform pull request.
Review critique on the PR and get the issue closed.

Next, I worked on points 1 and 2. the goal of this session was to simply get the bot to spit out information that I coded exactly as I said above.

I decided to copy over the code from the next command because it seemed like the most straightforward - it required no arguments from the user so I could quickly get to know how a discord bot truly works.

So how does a discord bot's commands work? Basically, in your index.js file (here called MHTimer.js), you create a collection and then just populate it with the contents of the commands folder using the following code. It's actually quite simple.

const Discord = require('discord.js');
const client = new Client({ disabledEvents: ['TYPING_START'] });
client.commands = new Collection();
 const commandFiles = fs.readdirSync('src/commands').filter(file => file.endsWith('.js'));
for (const file of commandFiles) {
    try {
        const command = require(`./commands/${file}`);
        if (command.name) {
            if (typeof(command.canDM) === 'undefined') {
                command.canDM = true;
                Logger.log(`Set canDM to true for ${command.name}`);
            }
            if (command.initialize) {
                command.initialize().catch((err) => {
                    Logger.error(`Error initializing ${command.name}: ${err}`);
                    throw err;
                });
            }
            client.commands.set(command.name, command);
        } else {
            Logger.error(`Error in ${file}: Command name property is missing`);
        }
    } catch (e) {
        Logger.error(`Could not load ${file}:`, e);
    }
}
Logger.log(`Commands: Loaded ${client.commands.size} commands: ${oxfordStringifyValues(client.commands.map(command => command.name))}`);

Please be aware that this code is from MHTimerBot and is not at all universally applicable. See the Discord docs for more information.

This session was pretty short so I also decided to figure out what the author of the issue had in mind when they specified "Can use the loot nicknames used by ifind."

const all_loot = getLoot(searchString, message.client.nicknames.get('loot'));
        if (all_loot && all_loot.length) {
            // We have multiple options, show the interactive menu
            urlInfo.qsParams = opts;
            sendInteractiveSearchResult(all_loot, message.channel, formatLoot,
                ['dm', 'group'].includes(message.channel.type), urlInfo, searchString);
            theResult.replied = true;
            theResult.success = true;
            theResult.sentDM = ['dm', 'group'].includes(message.channel.type);

I believe this is the code block they're referring to when they said that so I'll try to incorporate that when I'm writing the command in my second session.

Session Two

I started off the second session with a bit of a roadblock.

Ending the last session on what was basically a baseline command may not have been a mistake but it did mean I had a lot of work to get done. The roadblock came from not really feeling like I knew how the discord bot operated. After fumbling about for half an hour trying to figure out how to write my command by just staring at the others, I decided to map out exactly what my command needed to do, and where, in the code, it needed to go.

message -> 

getConvertibles() [match search term with database items] -> 

formatConvertibles() [get possible items and quantity, display in columns]

This is a rough line I want my command to go through. I also mirrors the line the find command for mice and loot takes.

After figuring this out, I could start on my coding.

Main code from whatsin.js

const theResult = new CommandResult({ message, success: false, sentDM: false });
    let reply = '';
    const opts = {};
    const urlInfo = {
        qsParams: {},
        uri: 'https://agiletravels.com/converter.php',
        type: 'convertible',
    };
    if (!tokens)
        reply = 'I just cannot find what you\'re looking for (since you didn\'t tell me what it was).';
    else {
        const searchString = tokens.join(' ').toLowerCase();
        const all_convertibles = getConvertibles(searchString, message.client.nicknames.get('mice'));
        if (all_convertibles && all_convertibles.length) {
            // We have multiple options, show the interactive menu
            urlInfo.qsParams = opts;
            sendInteractiveSearchResult(all_convertibles, message.channel, formatConvertibles,
                ['dm', 'group'].includes(message.channel.type), urlInfo, searchString);
            theResult.replied = true;
            theResult.success = true;
            theResult.sentDM = ['dm', 'group'].includes(message.channel.type);
        }

This is pretty close to what the find.js or ifind.js looks like. Since what I'm trying to do is work with the same api that command is in order to do almost the exact same thing, I believed I could reuse a lot of that code.

I had a few setback this session, but most were easy to deal with. The only one that gave me trouble was resolving in issue with the .map() function. I was accidentally using a filter when I didn't need to.

Excerpt from formatConvertibles()

const converter = results
        .map(convertible => {
            return {
                item: convertible.item.substring(0, 20),
                average_qty: convertible.total_items / convertible.total,
            };
        });
    const order = ['item', 'average_qty'];
    const labels = { item: 'Item', average_qty: 'Average Qty' };

At the end of the second session, I feel very confident and am ready to begin work on the tests next session.

Session 3

This is the last session, or at least, that last planned session.

All that this session consisted of was me cleaning up the code a bit and sending the pull request off to the main developers.

That said, if there are any changes they need me to make, or if they ask me to add tests, I will document the process here.

So what we need to do following this is 2 things: follow-up on creating the tests and fix our line endings. Two fairly simple things.

The code for the tests is written with sinon.js, a framework that I'm not terribly familiar with but shouldn't pose too much of a problem.

That said, I had a little trouble figuring out how I should go about doing my tests. Should I add tests in the mhct-lookup file to cover the new functions I built in the sister file? Or should I create my own test file for my new command? I figured I should do both.

Then, after working with the test files for hours I figured out that when I mock a function in one file, and it's mocked in another, it completely demolishes the second file and breaks the test. I needed to either scale down my testing or find some kind of workaround.

Creating the First Release and Link Checker 1.0

Ray — Sun, 06 Dec 2020 04:17:14 +0000

For a few months now, I've been working on an app to check links in files. This app has grown with me through these months while I worked on other open-source projects and although I don't feel like it's perfect, I do feel like I achieved something really important to myself.

Today, I'm releasing Link Checker through npm and I can't wait to take well-deserved break from the work, but also to continue development on the app if only to clean it up and fix more bugs.

Creating a release actually isn't too difficult for Node cli developers. One must simply create an account on the npm website and start into publishing the app. I found hte process relatively painless, aside from figuring out how to connect my npm account to my local version.

Learning how to set up npm and publish these things will be invaluable in the future, especially if I create a Node package that I use in a different project - I absolutely needed to know these things and I'm so happy I learned.

For other users, it might be different, but I didn't alter my code too much. I spent some time changing my version number and reading up about package.json and how it affects my app but otherwise, I felt totally comfortable simply publishing the app and testing it out in other folders to a smooth release.

However, I ran into a snag when I contacted a friend of mine, who worked with me on our PRJ666 project and asked him to test out the app and let me know if he is confused at all.

Well, he wasn't. The instructions are pretty simple for a developer to understand! On Linux at least.

On Windows, however, it's not so easy.

Hmm... I wonder why that's the case...

So I spent some time looking up fixes and reasons why npm scripts might not play nice with Windows and eventually settled on... well that they just don't play nice. And without a kind of step-by-step instruction for changing environment variables and installing npm globally (which I'm not even sure is the problem), I don't think I'll have Windows working in the way I intended. So I figured out a little workaround. If the user installs Link Checker with npx, npm will then install the app and run it at the same time - essentially performing the node index.js from within its own little framework. This works well enough to run the program on windows and if my developer friend finds it easy to use - we can close the book on that one.

So here it is. Link Checker. You should be able to follow the instructions in the description but here's a little more in-depth information.

You must have some kind of text file. When I used this app on a docx file it took from it the links to the pictures used in the file but not the links - weird stuff but I want to take a look at that. Any file that has text in it will work, doesn't matter if it's a .log or a .src, if it has text, it works.
I'm fairly certain you would be able to use it in a Node project of your own through require() but it's not built for that and I have no documentation on how that would work. It's a good next step though.
Finally, as said in the description - just npm install l-checker and go wild. The description has everything you might need to know about how to use it and I'll update it as I add more features.

If you want to contribute, please visit the git repository and shoot me a pull request.

That's it for now, the culmination of a semester's worth of work and one of the most informative courses I've ever taken. It's not over though, tomorrow I'll be writing about my progress in working on my final assignment or Release 0.4, and at the end of next week I'll have a whole lot to write about what I learned in this course and how valuable I think all of it was.

For now. Peace.

Lab 8 and Testing Apps

Ray — Sat, 28 Nov 2020 01:59:26 +0000

Testing: the backbone of any project. A project without the ability to test its code will inevitably bloat in a way that breaks EVERYTHING.

I don't think I need to actually explain the philosophy, but I will anyway. Basically, you need tests to make sure no one breaks the project as they're working. When you have a project of any reasonable size, you can expect that someone will fix a bug or add a feature that causes a different part of your project to not work. So we need to automate tests that will check to make sure we haven't done that before we go to add the feature or bug fix.

So to that end, let's write some tests!

Jest test for printing a working url

test('print good url', async () => {
        const expected = chalk.green('[200] GOOD http://www.google.ca')
        const result = isValid.displayStatusCode(200, 'http://www.google.ca')

        expect(result).toEqual(expected)
    })

The Framework

So we decided to use Jest! It's a great framework for testing Javascript projects. To do this, we simply use npm install --save-dev jest and go off! I created a test folder and put my testing files in there.

Inside the testing file, we define the files we're using and start writing tests!

The Tests

The first test I wrote was the most difficult. Actually, that applies to every category of test. Because we're testing the same function - just different lines through - we need to write one good test, and we can kind of reuse it for our other lines.

However, there wasn't anything particular that stood out to me as supremely difficult.

No particular bugs either, though I did learn about how my code works a bit more and had to check for undefined in some tests (which is, forgive the rhyme, rather unrefined a solution). I also realized I had to modulate my code a bit and separate some functions into their own files. This will be extremely beneficial for the future in general.

The Mock Tests

One of the most important parts of my app is checking for a response from a website. We want to know if the website will return 200 good or 404 broken, you know? However, you can't really check a real site in tests because what if it goes down? That's what mocking is for.

A test function that mocks a link to always output 405 status code

const host = 'https://www.youtube.com'
const path = '/'

test('get 405 response for unknown url', async () => {
        nock(host).head(path).reply(405)
        const url = `${host}${path}`
        const expected = chalk.gray(`[405] UNKNOWN ${url}`)
        const result = await isValid.isValid(url)

        expect(result).toEqual(expected)
    })

We use nock here to easily mock an address and send it to our function.

Easy!

Code Coverage

Code coverage is a description of exactly how much of your code is being tested by your tests. For example, if I'm writing 1 test that goes through 1 line of a function while there are 3 possible lines, my code coverage is 1/3 (realistically more but for the purposes of explanation let's say 1/3). So I need to write some more tests to expand that.

Also, there are tools one can use to inspect how much code is being covered by our tests.

Jest has such a feature! pictured above, all you have to do is specify --coverage when you run jest.

package.json excerpt

"scripts": {
    "test": "jest --detectOpenHandles --coverage",
    "prettier": "npx prettier --write .",
    "lint": "npx eslint ."
  },

I think it would be good to set coverage as a separate option, but also I think it should run for any user who is submitting tests because who wouldn't want as much coverage as possible?

My test cases for my isValid file covered everything. I wanted to leave some for others to do if they decided so I left spots in the other files.

GitHub

Also, I was required to create a testing suite for GitHub - this way, anyone who wants to submit a pull request has to pass my tests!

Above is my YAML file. It's pretty straightforward and just runs my test command upon upload. I used the boilerplate settings for GitHub Actions and I think I'll expand on it in the future.

A Stranger's Tests

My partner's repo wasn't wholly different, and she used the same framework for testing (Jest) so I knew some of the syntaxes. There were marked differences though like that she used mock functions and I wasn't quite sure how they worked. So it took a little acclimation but once I found some work to do I got to it. Her regular program was different too and I don't think I realized how difficult it is to immerse yourself in a different code base and anticipate the
outputs from my testing inputs.

What I learned

So firstly, I have done testing a bit. I created tests for MHTimer and Telescope respectively so I'm not a stranger to tests.

However, working on my own tests gives me a sense of productivity that I hadn't realized before. It's really nice to set up and let grow as a project within the project.

I really like developing tests and think that it also helps me acclimate to other code bases and I for sure will do it in the future, especially for features and bugfixes I write.

Release 0.4 Defined and Colourblindness

Ray — Sat, 21 Nov 2020 20:12:03 +0000

This blog post is a living document of sorts, where I'm going to document my journey to finding the right issue to work on for Release 0.4. So, to that end, I'm going to take a look at all the projects David linked me and try to find something important to work on.

The Start

This week we got our first glimpse of what Release 0.4 would look like.

Due December 11th, the requirements for Release 0.4 are simple: do something larger than we've ever done before. Examples were a feature, a number of bug fixes (like 3), or a large bug fix in a large repo.

That's a lot to choose from and I think it's a great way to step up to another level in our open source journey.

What would I want to work on?

Ideally, I want to make a meaningful contribution in the form of a feature to a project. I think that's the next step that makes the most sense to me.

So far my experience in working on open source projects has been in fixing bugs or adding a feature that was meant to already be there like in MHTimer, cleaning up code, or moving from one type of authentication rule to another like in Telescope. I don't feel like I've contributed to a project in a way that I can be really proud of. Not to say I don't think my contributions so far have been good - I think they've helped me learn to get immersed in different code bases and styles and trace sequences of code through different modules and all that. And that's useful! But I really think adding a feature would round out my knowledge and give me something I can be really proud of, which is what I want to get out of this class.

This brings us to colorblindness, weirdly enough.

The idea

I don't find colorblindness to be a thing that defines me and I think acting like it is would be in poor taste toward people with more inhibitive accessibility issues, but I think as someone who's affected by deuteranopia, I can give some insight into how some IDEs highlight their code and how it can be exclusionary to colourblind people. And how to fix it!

For example, I can't tell the difference between these two colours unless they're blown up.

A high contrast vscode theme would greatly help someone like me - unfortunately, there's little to no extensions that do this.

If you're also colourblind, please check out https://marketplace.visualstudio.com/items?itemName=PedroFonsecaDEV.dark-plus-colorblind&ssr=false#overview and give him all your feedback.

The first issue

So, keeping in mind that I want to work ona feature for an app, I thought I would go through some of the old repos that I've worked on in the past. I found this issue in the MHTimerBot

Loot Plurals should be squashed some of the time #212

AardWolf posted on Oct 09, 2020

searchByItem now returns singular and plural names for loots. So a search for "cloud curd" matches cloud curd and cloud curds. They both point to the same time but the result menu makes it look like there are two matches.

View on GitHub

While I'd like to work on it, and it's not labeled a bug, I think I would consider this a bug and would therefore probably not be in my purview of work. However, I think if I really wanted to, I could work on this and be fulfilled in some way. so it's an option.

Add Ability To Find Things in Shops #63

AardWolf posted on Mar 02, 2018

Mrs Flibble / kipper has offered to help.

Something like current find query but that would show things like: charmbits are buyable for X gold in ... charmbits can be dropped in .... (location) charmbits can be looted from ... (convertible)

View on GitHub

The issue above is another one I'm thinking of taking up, but I think it's already being worked on? Or perhaps in on hold - it's from 2018 after all. I wouldn't want to step on any toes but it's been so long that I think maybe it could be something I work on.

Release 0.3 part 2, Telescope

Ray — Thu, 19 Nov 2020 02:56:07 +0000

The second task for our Release 0.3 was to fix and issue and merge a pull request into telescope. This involves finding an issue, fixing it, collaborating with the telescope slack channel, and finally getting feedback and eventually merging the pull request.

This sounds like a lot of steps but depending on the issue, they can take very little time and produce very little headache.

This is not the case with mine.

The issue

So my issue was this:

Add query params validation for DELETE /feeds/:id route using Express Validator and unit tests #1308

tonyvugithub posted on Nov 11, 2020

What would you like to be added: Request to add query params validation for DELETE /feeds/:id route using Express-validator and also unit tests to ensure that it works

Why would you like this to be added: It would be better to validate the query before it reach the route handler.

reference: https://express-validator.github.io/docs/check-api.html

All logic for the middleware should be place inside backend/web/validation.js

View on GitHub

Basically, I'm meant to go into the backend for Telescope and add some query parameters with express-validator. Then I have to link them to the DELETE feed route and write tests for it.

First, what I did was add a new validation parameter to the validation.js, but after some review, we decided it was better to use what already existed to not bloat our code.

validation.js excerpt

const feedsIdParamValidationRules = [
  param('id', 'Id Length is invalid').isLength({ min: 10, max: 10 }),
];

module.exports = {
  validateNewFeed,
  validateQuery: () => validate(queryValidationRules),
  validatePostsQuery: () => validate(postsQueryValidationRules),
  validatePostsIdParam: () => validate(postsIdParamValidationRules),
  validateFeedsIdParam: () => validate(feedsIdParamValidationRules),
};

A fairly simple piece of code that establishes a rule that an id must be a minimum 10, and a maximum 10 (aka exactly 10) characters long.

Then, in our feeds.js file, which holds our routes for blog feeds, we have to make sure our delete feeds route checks our new rule.

delete feeds route

feeds.delete('/:id', validateFeedsIdParam(), protect(), async (req, res) => {
  const { user } = req;
  const { id } = req.params;
  try {
    const feed = await Feed.byId(id);
    if (!feed) {
      return res.status(404).json({ message: `Feed for Feed ID ${id} doesn't exist.` });
    }
    if (!user.owns(feed)) {
      return res.status(403).json({ message: `User does not own this feed.` });
    }
    await feed.delete();
    return res.status(204).json({ message: `Feed ${id} was successfully deleted.` });
  } catch (error) {
    logger.error({ error }, 'Unable to delete feed to Redis');
    return res.status(503).json({
      message: 'Unable to delete to Feed',
    });
  }
});

Notice the promise call that includes validateFeedsIdParam(), this is our rule that will be tested against when we submit an id to the delete request.

the test

it('should return 400 when id is less than 10 characters', async () => {
    const id = '123456789';
    const res = await request(app).delete(`/feeds/${id}`);

    expect(res.status).toEqual(400);
  });

  it('should return 400 when id is more than 10 characters', async () => {
    const id = '123456789123456789';
    const res = await request(app).delete(`/feeds/${id}`);

    expect(res.status).toEqual(400);
  });

keep in mind this is after a multitude of critique.

Oops!

so now that we have our code working, our test working properly, all that's left is to merge into the project right?

Not quite. That's pretty easy, let's just rebase on GitHub since it's only one line that seems to be wrong!

Oops. Again.

Ok ok ok this is fixable, all I have to do is go back, run some commands, fix up in an interactive rebase, and commit again.

For those that don't know, here's a nice image that'll help you out if you end up in my situation.

Wait a sec, it looks like something else is wrong...

Huh?

Ok. So here's what happened. I go ahead and run prettier on my files but I ran it through vscode, which means it's slightly different from the Telescope configuration for prettier. Now you would expect a test to check this, find it wrong, and send back an error, right? But no, it actually thinks everything is good!

So what is there to do?

Be more explicit with prettier config #1399

humphd posted on Nov 18, 2020

We ran into a weird issue where Prettier was not seeming to use some of our values which are implicit defaults. This sets all prettier options explicitly.

View on GitHub

So David opened up a pull request to fix the prettier config and make sure this doesn't happen again, but I'm still in a position where my code is all messed up. So I guess the only thing to do is manually fix our formatting errors.

I'll spare you the screenshots.

Finally, we've got our pull request. Formatting? Great! Functionality? Awesome. Tests? Work properly! Now all we have to do is commit, rebase PROPERLY, and push!

Yes!

So to conclude - this has been a very educational experience. Not just in Git and how to work with a project's contributing guidelines (and what happens when you mess it up), but how to interact in a community and get feedback from members. Until now all my PRs have been basically some sequence of being assigned, working, putting up a PR, getting little to no feedback, and instantly having it merged. This experience really helped me figure out how to take criticism and build on my first would-be contribution.

Release 0.3 part 1, more Discord!

Ray — Wed, 18 Nov 2020 02:00:55 +0000

For the first part of Release 0.3, I decided to revisit my favorite repo from last month: MHTimer!

If you're new to Discord bots or have started reading with this post, I recommend checking out my other post about this bot and how I got it working as I won't be going over it again.

This time I took on an issue that had to do with error handling.

findThings sends empty response on error. #210

AardWolf posted on Oct 01, 2020

findThings: Error getting item item_type=mouse&item_id=402 - FetchError: invalid json respons

View on GitHub

So basically when the bot gets a request for some information, and the website is down, the bot just prints an error in the terminal and the Discord chat has no printout!

So the first step was to take a look at the get call.

findThing function

async function findThing(type, id, options) {
    if (!type || !id)
        return [];

    // If caching is ever implemented it'd be checked here
    const qsOptions = new URLSearchParams(options);
    qsOptions.append('item_type', type);
    qsOptions.append('item_id', id);
    const url = 'https://www.agiletravels.com/searchByItem.php?' + qsOptions.toString();
    return await fetch(url)
        .then((response) => {
            if(response.ok){
                return response.json();
            }
            else {
                return null;
            }
        })
        .catch(err => {
            Logger.log(`findThings: Error getting item ${qsOptions.toString()} - ${err}`);
        });
}

So what this did before simply sent back an error when any problem happened. Instead, I decided to return null if the response from the server was not ok. Next, let us take a look at the function that will craft our message.

formatMice function

async function formatMice(isDM, mouse, opts) {
    const results = await findThing('mouse', mouse.id, opts);
    if (results === null) {
        const reply = 'Looks like I\'m having a bit of trouble finding your mouse right now.' ;
        return reply;
    }
    const no_stage = ' N/A ';
    const target_url = `<https://www.agiletravels.com/attractions.php?mouse=${mouse.id}&timefilter=${opts.timefilter ? opts.timefilter : 'all_time'}>`;
    const attracts = results.filter(mouse => mouse.total_hunts > 99)
        .map(mice => {
            return {
                location: mice.location.substring(0, 20),
                stage: mice.stage === null ? no_stage : mice.stage.substring(0, 20),
                cheese: mice.cheese.substring(0,15),
                total_hunts: intToHuman(mice.total_hunts),
                ar: mice.rate / 100,
            };
        });
    if (!attracts.length)
        return `There were no results with 100 or more hunts for ${mouse.value}, see more at ${target_url}`;
    const order = ['location', 'stage', 'cheese', 'ar', 'total_hunts'];
    const labels = { location: 'Location', stage: 'Stage', total_hunts: 'Hunts',
        ar: '/Hunt', cheese: 'Cheese' };
    //Sort the results
    attracts.sort((a, b) => parseFloat(b.ar) - parseFloat(a.ar));
    attracts.splice(isDM ? 100 : 10);
    if (attracts.every(row => row.stage === no_stage))
        order.splice(order.indexOf('stage'), 1);
    // Column Formatting specification.
    /** @type {Object <string, ColumnFormatOptions>} */
    const columnFormatting = {};
    const headers = order.map(key => {
        columnFormatting[key] = {
            columnWidth: labels[key].length,
            alignRight: !isNaN(parseInt(attracts[0][key], 10)),
        };
        return { 'key': key, 'label': labels[key] };
    });
    // Give the numeric column proper formatting.
    // TODO: toLocaleString - can it replace integerComma too?
    columnFormatting['ar'] = {
        alignRight: true,
        isFixedWidth: true,
        suffix: '%',
        columnWidth: 7,
    };
    let reply = `${mouse.value} (mouse) can be found the following ways:\n\`\`\``;
    reply += prettyPrintArrayAsString(attracts, columnFormatting, headers, '=');
    reply += '```

\n' + `HTML version at: ${target_url}`;
    return reply;
}

It's a bit long, but let us take a look at the most important part.



const results = await findThing('mouse', mouse.id, opts);
    if (results === null) {
        const reply = 'Looks like I\'m having a bit of trouble finding your mouse right now.' ;
        return reply;
    }

Basically, what we do here is call our findThing function, which this function was already doing, and if our return value is null, then we craft our message and skip the rest of the validation and separation. Who needs it if we know our response is bunk?

So let's put up our fix for review.

Whoops! Turns out I ran prettier on my code before submitting and now it's gone and messed up tons of stuff. Let's patch it up and submit it properly.

We've succeeded! All in all some good work. To be honest, I didn't learn a lot doing this fix. I became more familiar with the project and the features of git - but I think what really shows my growth was how easily I was able to take an issue, fix it up, and merge it quickly! I really feel like I've progressed as a developer because of that.

Lob 7, the tools I'll never do without

Ray — Sat, 14 Nov 2020 21:14:28 +0000

This lab was all about setting up static analysis tools. Specifically, I set up Prettier and ESLint.

Prettier

Prettier is a tool for Javascript that automatically organizes code to look uniform. It's nothing terribly special, though it does allow the user to specify a lot of options for how it should look. I didn't really set any though because I feel like getting used to a certain style that's not like the default would hinder me in working on other projects in the future.

To be perfectly honest, choosing Prettier was just due to the fact that it was my first choice and I had seen it used all over the place. This week's lecture also spent a lot of time talking about Prettier so I thought I should get to know it.

ESLint

ESLint is a tool for fixing minor syntax errors in your code. It's meant to be able to keep the code free of little problems that could end up breaking something in the future. Lint!

Again, because ESLint is widely known and used I picked it because I wanted to get to know it and use that knowledge in the future.

Getting it started

For Prettier it was fairly easy. I followed the install page and ran the first command in my command line.

npm install --save-dev --save-exact prettier

and likewise created my prettier config file.

echo {}> .prettierrc.json

After this, I could add some custom styling in the config file but as I said above I decided not to do that.

As for ESLint, I took similar steps following the ESLint getting started page

npm install eslint --save-dev

npx eslint --init

All good! We can now lint any file we like with ESLint.

Before I move on I also wanted to mention eslint-config-prettier because according to Prettier, this helps the two tools work together and not break each other.

User experience

Code after using Prettier

if (options.t) {
    var url = 'http://localhost:3000/posts/'

    req.get(
        {
            url: url,
            json: true,
            headers: {
                'User-Agent': 'request',
            },
        },
        (err, res, data) => {
            if (err) {
                console.log('Error:', err)
            } else if (res.statusCode !== 200) {
                console.log(
                    'localhost did not respond... check to see if your local telescope session is up...',
                )
            } else {
                const urls = data.map((e) => url + e.id)

                fs.writeFile(
                    'files/telescope.txt',
                    JSON.stringify(urls),
                    function (err) {
                        if (err) throw err
                    },
                )
            }
        },
    )
    //send file to be read
}

As you can see, the code looks a lot more readable! Prettier basically made all my code look like this and it's pretty great. The best part is I didn't even have to touch it! I clicked the button and my code was prettified.

ESLint

Running ESLint with npx eslint index.js revealed something...

So ESLint caught the fact that we wrote createWriteStream and never used it! Great. Let's just remove it and move on.

No other errors either! Which is pretty nice. Makes me feel pretty good about my code.

The command line

So, moving on from our first use of these tools, we want to allow the users to run a simple command and get these running on their own code.

package.json

"scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "prettier": "npx prettier --write .",
    "lint": "npx eslint ."
  },

Here we can see that I've written some scripts in my package.json so when the user inputs "prettier" or "lint" those tools will run! It's a fairly simple way to use the tools but there is another, even more consistent, way to get the user to run these scripts that I'll show in a moment.

IDE Integration

VSCode is a very useful IDE because you can just move over to the extensions tab and choose whatever you like, including Prettier and ESLint! You can even set shortcuts to run them on the IDE level.

For new users however, I decided to recommend they install Prettier and ESLint.

.vscode/extensions.json

{
    "recommendations": [
        "dbaeumer.vscode-eslint",
        "esbenp.prettier-vscode"
    ]
}

I'm not 100% sure how it works but I believe with this file, any other user that wants to work on Link Checker in VSCode will be asked to install these extensions.

(Optional)

I also went a little step further because I wanted to learn how to do this. I set up Prettier and ESlint in a pre-commit hook.

package.json excerpt

"husky": {
    "hooks": {
      "pre-commit": "lint-staged"
    }
  },
  "lint-staged": {
    "*.js": "eslint --cache --fix",
    "*.{js,css,md}": "prettier --write"
  }

you can see here that lint staged and husky are working together to run ESLint and Pettier before staging any commits. This means that no matter what, all commits will adhere to the linter and style guidelines! Next, I'd like to learn how to set up GitHub tests to test pull requests. If you'd like to add a pre-commit hook I recommend this prettier link.

Conclusion

All in all, this week was very informative and I'm going to be taking these tools to every project I create and am a part of. I basically learned how useful linting and code formatting tools are and how to integrate them so that they automate themselves through pre-commit hooks. I felt that was probably the most important thing I learned this week and I'll probably come back to this blog post every time I start up a new project because this will be one of the first things I set up.

Lab 6 After a Break

Ray — Sat, 07 Nov 2020 02:55:11 +0000

Well not much of a break.

This week was filled with a couple of firsts. The first week of my new job and the first time using most of the programs introduced this week. It was also the first time I'll be working with Telescope and the first time using git diffs and GitHub Gist. Both of which I'm not terribly sure if I have a full grasp of yet.

The Set Up

So for Lab 6, we were meant to set up Seneca's Telescope blog aggregator and to pump the links we get into our Link Checker apps. At least, we were meant to develop a proof of concept for the feature and upload the diff to GitHub Gist.

Seems easy, right? Well, it requires a little more effort than that.

In order to set up Telescope we need 3 things: redis-server, Elasticsearch, and Docker. In order to properly run redis-server, we must also get WSL, a Linux subsystem for Windows.

WSL was a slight pain to get working but now that I have it I think I might use it forever. Running a dual boot system for a while just to develop web apps because so cumbersome that I gave up early last year and just used Windows. But with WSL I can run a Ubuntu terminal in vscode and kill two birds with one stone. The only thing I have to look out for is accidentally having like 5 terminals open at once which... I'm doing right now to gather screenshots for this blog post.

So redis-server is up, next is Elasticsearch. Now, I'm not too sure where I went wrong here but after properly installing Elasticsearch and getting it running properly, as well as verifying that all the ports were the same between them, I just couldn't get Elasticsearch to properly connect with the Telescope backend. This is unfortunate because I'd really like to get it properly working before I take a dive into Telescope for real. However, for just this lab, I was able to set up a limited Elasticsearch server to grab some blog posts from the API and help me with my work.

Finally, Docker. I'm not too sure where Docker fits into everything because I ran everything separately. However, I think I'll try to get it all together in a Docker container for the future as it seems like it'll make development a lot easier. I had an issue with setting up Docker with my WSL that involved having the Hyper-V VM running which made Docker crash. Weird stuff, but after a bit of research, it was easy to fix.

A bit of coding

The objective is this: grab links to blog posts from the Telescope backend API, pipe them into a text file, grab the links from that text file, and finally test them to make sure they're good links like my app already does.

Simple, right?

This is the part where I say no, but actually it is.

index.js excerpt

if (options.t) {

    var url = 'http://localhost:3000/posts/';

    req.get({
        url: url,
        json: true,
        headers: {
            'User-Agent': 'request'
        }
    }, (err, res, data) => {
        if (err) {
            console.log('Error:', err);
        } else if (res.statusCode !== 200) {
            console.log('localhost did not respond... check to see if your local telescope session is up...');
        } else {
            const urls = data.map(e => url + e.id)

            fs.writeFile('files/telescope.txt', JSON.stringify(urls), function (err) {
                if (err) throw err;
              }); 
        }
    });
}

The above code basically queries the local version of the Telescope API for the post ids, then we just stick the rest of the url to the front of the id, then write it straight into a file.

more index.js excerpts

if(options.t){
    filename = 'files/telescope.txt'
}
else {
        filename = `${argv[2]}`;
}
fs.readFile(filename, (err, fileContents) => {
    try {
        var linkList = generateLinkList(fileContents);
        linkList = Array.from(linkList);
    } catch (err) {
        console.log("The app has recieved a wrong filename.")
        console.log("Please enter a correct filename.")
        exit(1);
    }
    validateLinks(linkList)
})

This is also a simple change to the rest of the code, writing an if statement to check if we've used our telescope argument, and setting the file to that if so! Easy.

Frankly, I just had to add some connections to the local API and modify almost nothing else.

I want to be clear that this code is a proof of concept as outline in the Lab 6 instructions, it's not mean to be comprehensive and perfect for every situation (for example, I could have it query a backend API that I host on a Heroku server or something so that I won't have to set up the local server every time to use it), but it works well for what it is. I can't wait to expand upon this feature and fix up the code to be a lot more modular, which I think is going to be my next big refactor.