DEV Community: Danny Guo

How to Fix instanceof Not Working For Custom Errors in TypeScript

Danny Guo — Wed, 12 May 2021 00:00:00 +0000

In JavaScript, you can create custom errors by extending the built-in Error object (ever since ES 2015).

class DatabaseError extends Error {}

You can do the same thing in TypeScript, but there is an important caveat if your tsconfig.json has a compilation target of ES3 or ES5. In that case, instanceof doesn’t work, which breaks any logic that is based on whether or not an error is a case of the custom error.

class DatabaseError extends Error {}

const error = new DatabaseError("Unique constraint violation");

// prints "true"
console.log(error instanceof Error);
// incorrectly prints "false"
console.log(error instanceof DatabaseError);

You can try this out yourself in the TypeScript playground. This is a known issue that started with TypeScript version 2.1. The recommended fix is to manually set the prototype in the constructor.

class DatabaseError extends Error {
    constructor(message: string) {
        super(message);

        Object.setPrototypeOf(this, DatabaseError.prototype);
    }
}

const error = new DatabaseError("Unique constraint violation");

// both print "true" now
console.log(error instanceof Error);
console.log(error instanceof DatabaseError);

Any custom errors which further extend DatabaseError still need the same adjustment.

class DatabaseError extends Error {
    constructor(message: string) {
        super(message);

        Object.setPrototypeOf(this, DatabaseError.prototype);
    }
}

class DatabaseConnectionError extends DatabaseError {
    constructor(message: string) {
        super(message);

        Object.setPrototypeOf(this, DatabaseConnectionError.prototype);
    }
}

const error = new DatabaseConnectionError("Invalid credentials");

// all print "true"
console.log(error instanceof Error);
console.log(error instanceof DatabaseError);
console.log(error instanceof DatabaseConnectionError);

Upgrade the Compilation Target

Remember that this is only an issue if your compilation target is ES3 or ES5. Instead of having to remember to set the prototype, you could consider upgrading your target to ES 2015 or even later. ES 2015 has over 97% browser support, so it may be a reasonable choice for you, especially if you are okay with dropping support for Internet Explorer.

What I Learned by Relearning HTML

Danny Guo — Fri, 07 May 2021 00:00:00 +0000

I’ve worked on websites for several years, both professionally and for side projects. One day, I reflected on the fact that all of my web development education had come from actually making websites. In most cases, I’d have a specific problem, Google how to solve it, and learn something new in the process.

I wondered what I was missing by never learning HTML in a comprehensive way. Forget CSS and JavaScript. I’m just talking about raw HTML. It might seem silly to go back to such a basic aspect of web development after a decent amount of experience, but it’s easy to become overconfident with a skill just because you know enough to do a few useful things.

So I decided to relearn HTML and discover my unknown unknowns.

Experience

For context, I made my first website in middle school for a class project. We learned basic HTML, and embedding a MP3 song felt like magic. But I didn’t touch web development again until college. I made a lightweight news aggregator called The Daily Lore that’s still running (I preserved the original version).

Since then, I’ve worked on two websites professionally, one website for a nonprofit, my personal website, and a few small websites for side projects, such as Make a README.

Introduction to HTML5

I don’t consider myself to be a web development expert based on just that experience, but I surely had far more knowledge than the typical student for Coursera’s Introduction to HTML5 course. I started the course expecting to know a lot of the content already, since it was designed for complete beginners with no programming backgrounds.

As I went through the material, I did in fact know a lot of it already, but it was still a good refresher for two points in particular: the importance of using semantic elements and what to think about in terms of accessibility.

I’ve always had a bad habit of using generic <div> elements to make what I need, rather than semantic elements that represent specific content, like the header and footer elements.

Accessibility was also something I had never considered in depth. I knew that images should have alt descriptions, and that was about it. One of the course’s key points is that using the appropriate semantic elements is important to making a site more accessible.

For example, people who use screen readers can jump around using heading elements (<h1> through <h6>), so it’s important to use them and make sure they’re in the correct order. It’s wrong to use them only to make text bigger because their real purpose is to define the structure of the content. They’re like a table of contents.

Instead of headings, we could use <p> elements and alter their font sizes with CSS to create a website that looks identical, but it’d be less semantic and less accessible. There is more to web development than making websites look the way we want. It’s important to make the content mean what we want as well.

Accessibility isn’t just about improving how websites work with screen readers. We should think about font size, font style, and color contrast for people who have visual impairments or color blindness. We should consider that people who have hearing loss may have a harder time recognizing that audio or video is playing. We should make tab navigation work well for people who rely primarily on the keyboard, perhaps because they have a difficult time using a mouse. When we add animations, we should take care to avoid ones that make it more difficult for someone to actually use the website, such as animations that change the page layout in the middle of interactions. And we should consider when a page is overloaded with too much information or too many elements, making it hard for people to understand things or how to actually use the website.

It’s easy to forget about accessibility, but we should strive to make websites work well for as many people as possible. Accessibility also goes hand in hand with usability and search engine optimization. The course points out that improving one frequently means improving all the others.

Reading the Documentation

I have a friend who is probably the only person I know who has read the entire NFL rulebook (the 2020 version is 87 pages long). Watching football with him was fun because he was so good at understanding nuances to the game and weird situations. I figured there was a similar opportunity for me with HTML.

The strict equivalent would have been to read the HTML standard for every HTML element, but I decided to read the MDN documentation for every element instead since MDN has a lot of information about browser compatibility and using elements in practice. I read the entire page for each element, took notes, and made Anki cards for the bits that I wanted to commit to memory.

There were many deprecated elements that I only skimmed through, and I didn’t bother to take notes for those, but dozens of standardized elements and attributes were totally new to me.

I didn’t intend to come out of this experience as a master of HTML, and I still have to apply what I’ve learned (including to this website), but I find it useful just to be aware of what is available. Even though I can’t recall all the details about using a picture element, I know it exists now, and I can always look up the details later during implementation. It’s a categorical difference from not being aware of it at all and using a plain <img> for all cases because I don’t know any better.

Observations

As I read the documentation, some things were particularly interesting to me, and I had some observations.

The address element is for contact information in general, not just physical mailing addresses.

The definition element represents the term that is being defined, rather than the definition itself.

There is a whole set of ruby elements that are primarily used to show the pronunciations of East Asian characters.

The track element provides a standard way to embed timed text tracks for video and audio. I had never heard of the WebVTT (Web Video Text Tracks) format before.

The map element seems like an anachronism, especially considering that it isn’t responsive.

The data element provides a machine-readable translation for content. This seems likes it could help screen scraping, which some websites like LinkedIn have been actively trying to prevent.

There’s subtlety when it comes to correctly choosing to use strong versus em versus i versus u versus b versus mark.

There are a few elements that seem redundant. The legend element represents a caption for a fieldset element, the caption element represents a caption for a table element, and the figcaption element represents a caption for a figure element. I don’t know why one element couldn’t do the job for all three, since the meaning could be derived from the parent element.

The Future of HTML

As I read through the documentation, it kept making me consider the question of how HTML should evolve. Browsers keeps gaining more and more functionality, to the point that they are becoming more like operating systems. There’s even an experimental API for connecting to Bluetooth devices.

Wikipedia is the perfect website for what HTML was originally designed for: mostly static documents that are connected through hyperlinks. But now we use the browser to deliver full on applications, like Figma, which is a design tool that effectively runs C++ code in the browser by compiling it to WebAssembly.

HTML has added a few elements and attributes that make interactivity possible without JavaScript. For example, the details element creates a widget that can be toggled between open and closed states.

But as your use case become more advanced, it quickly becomes difficult to rely solely on what HTML provides. For example Bootstrap’s progress bar doesn’t use the HTML progress element.

We don’t use the HTML5 <progress> element, ensuring you can stack progress bars, animate them, and place text labels over them.

Another example is the table element. Pure HTML tables can be pretty sophisticated in terms of displaying data, but there’s no built-in support for interactive functionality like sorting, filtering, and pagination.

Browser support also becomes an issue when an element does become more advanced. The input element is one of the most complex elements because it supports so many combinations of input types and attributes. In theory, you could use it to easily collect a date and a time, using the datetime-local type. But not all browsers support it, and there is variation in how it works even among those that do.

Some elements are also difficult to style, such as the select element. So website developers may want to rely on standard functionality instead of using a library or implementing a feature themselves, but then they have to worry about it not working well in certain browsers or stylistic inconsistency with the rest of the website.

I’m eager to see if Web Components become more popular and provide a good solution to these problems. If they do, the situation could become similar to programming languages, where different languages take difference stances on the question of how much functionality should be included in the standard library (HTML) so that the community has a lesser or greater tendency to rely on third party libraries (Web Components).

Web Components do seem to be picking up some momentum. GitHub has started to use them, and they publish their components to WebComponents.org.

Conclusion

It was easy to feel confident with HTML after doing web development for several years. Yet I found plenty of value in going back to learn it in a more rigorous manner. I learned about many improvements I can make to the websites I work on, and I have a better big picture view of HTML and how it might develop. While I still think learning by doing is highly effective, this experience has made me want to go back and relearn other things with a bottom up approach.

How to Concatenate Strings in Lua

Danny Guo — Tue, 29 Dec 2020 00:00:00 +0000

The most straightforward way to concatenate (or combine) strings in Lua is to use the dedicated string concatenation operator, which is two periods (..).

message = "Hello, " .. "world!"
-- message equals "Hello, World!"

Numbers are coerced to strings. For fine-grained control over number formatting, use string.format, which behaves mostly like C's printf.

count = 42
message = "The count is: " .. count
-- message equals "The count is: 42"

Trying to concatenate other types, like nil or a table, will result in an error.

count = nil
message = "The count is: " .. count
-- results in an "attempt to concatenate a nil value" error

Note that Lua doesn't have syntactic sugar for augmented assignment. The following is invalid syntax.

message = "Hello, "
message ..= "world!"
-- results in a "syntax error near '..'" error

Strings in Lua are immutable, so the concatenation result (message in this example) is a brand new string.

start = "Hello, "
message = start .. "world!"
start = "Bye, "
-- message still equals "Hello, World!"

table.concat

If you need to perform many concatenation operations, using the concatenation operator can be slow because Lua has to keep reallocating memory to create new strings.

message = ""
for i=1,100000 do
  message = message .. i
end

As a result, it can be much faster to use table.concat.

numbers = {}
for i=1,100000 do
  numbers[i] = i
end
message = table.concat(numbers)

Here's a benchmark comparsion (using hyperfine) from running the .. example as slow.lua and running the table.concat example as fast.lua.

hyperfine 'lua slow.lua'
# Benchmark #1: lua slow.lua
#   Time (mean ± σ):      1.287 s ±  0.115 s    [User: 1.120 s, System: 0.078 s]
#   Range (min … max):    1.187 s …  1.528 s    10 runs
hyperfine 'lua fast.lua'
# Benchmark #1: lua fast.lua
#   Time (mean ± σ):      39.3 ms ±   3.8 ms    [User: 34.6 ms, System: 2.8 ms]
#   Range (min … max):    35.3 ms …  58.3 ms    48 runs

The difference probably doesn't matter in most cases, but it's a good optimization to be aware of.

table.concat can also be easier to use because it can take a separator argument to add between elements.

message = table.concat({1, 2, 3, 4, 5})
-- message equals "12345"
message = table.concat({1, 2, 3, 4, 5}, ", ")
-- message equals "1, 2, 3, 4, 5"

It can also take start and end indexes. Keep in mind that Lua arrays start with index 1.

message = table.concat({1, 2, 3, 4, 5}, ", ", 2, 4)
-- message equals "2, 3, 4"

Direct Approach

Depending on your use case, you might be able to save some memory usage over table.concat by generating the result directly.

for i=1,100000 do
  io.stdout:write(i)
end

Fixing MacBook Pro Thermal Performance Issues

Danny Guo — Sun, 02 Aug 2020 00:00:00 +0000

I may earn commissions from purchases made through affiliate links in this post.

Over the past several months, my 2018 15" MacBook Pro developed thermal performance issues. Here's what I did to fix the problem. The two key changes were that I switched to using the right side Thunderbolt 3 ports instead of the left side ones, and I cleaned the dust buildup in the internal fans.

The Problem

The problem first started when I noticed my CPU usage spiking to near 100%. In the process list, I saw kernel_task taking up most of the CPU.

I learned that macOS does this intentionally to prevent the CPU from overheating.

One of the functions of kernel_task is to help manage CPU temperature by > making the CPU less available to processes that are using it intensely. In > other words, kernel_task responds to conditions that cause your CPU to become > too hot, even if your Mac doesn't feel hot to you. It does not itself cause > those conditions. When the CPU temperature decreases, kernel_task > automatically reduces its activity.

Fan Speed

For many years, I've used smcFanControl to manually increase my fan speed because I don't mind fan noise, especially if it means getting better performance out of my laptop or being able to comfortably put it on my lap.

However, it stopped working reliably after a macOS update. I purchased iStat Menus, which comes with a well supported fan control feature. Note that you have to buy it from their own website to get fan control. The Mac App Store version doesn't include that particular feature for some reason.

I turned the fans up to max regularly in an attempt to preempt the kernel_task issue, but it still kept popping up.

Thunderbolt 3 Ports

I found this excellent StackExchange answer that recommends using the right side Thunderbolt 3 ports instead of the left side ones. Apparently the ports aren't equal. The left ones can cause thermal issues. Also, using both ports on either side can be problematic.

I use a Thunderbolt 3 dock for a single cable solution that handles charging and all my peripherals, including monitors. I tended to plug it into a left port. I switched it over to a right port and saw an immediate difference in temperature. iStat Menus reports the temperatures from various sensors.

I tested and verified that using a left port caused the CPU temperature to go up much higher than using a right port.

For a few months, this simple change was enough to resolve the issue. Luckily, I was able to continue using a single cable, and I didn't have to plug in my dock on one side and then a separate charger on the other side.

Resolution Scaling

Then the problem started happening again. I also experienced issues playing Tabletop Simulator, Counter-Strike, and Valorant (on Windows through Boot Camp), all games that my machine should be able to handle without any problems. It would stutter even after I turned the resolution and graphical settings all the way down.

I thought it might have something to do with my monitors. In my desk setup, I have two external monitors that I run at a scaled resolution that looks like 2560 x 1440.

Note the warning that "Using a scaled resolution may affect performance." The problem is that macOS achieves this resolution by rendering everything in double the size and then scaling it down. So my MacBook is essentially doing the rendering work for two 5K monitors.

See this thread or this StackExchange answer for more details.

I considered turning the resolution down to look like 1920 x 1080, which isn't a scaled resolution, but then everything looked too big.

I was pretty frustrated at this point, especially because Zoom video calls started freezing up on me. I got into the habit of unplugging my monitors before joining video calls.

Dust

This entire time, I thought the problem was my MacBook producing too much heat. Beyond making the fans run at max more often, I never considered that the problem could be that it stopped being able to get rid of heat.

Then I read this post from someone who cleaned the inside of a MacBook after 7 years of use. I was initially skeptical that clogged fans were my problem because my MacBook was only about a year and a half old. But the more I thought about it, the more it made sense, and I didn't have anything to lose by trying to clean it.

To open my MacBook, I followed this iFixit guide.

AutoBoot

I learned that by default, macOS turns on the MacBook when you press any button (not just the power button) or open the lid. This was very surprising to me. I did recall being confused in the past when my MacBook turned on even though I didn't press the power button. When it first happened, I thought there was something wrong. At least from Apple's perspective, it turns out this is a feature, not a bug.

So to make sure the MacBook stays off through the process, you have to run this in a terminal to turn off booting on lid open.

sudo nvram AutoBoot=%00

To turn the setting back on later, run this. Though I just left mine off after I was done.

sudo nvram AutoBoot=%03

Removing the Bottom

The next step was to take off the screws. Apple uses pentalobe screws.

I've opened MacBooks before to replace swollen batteries, so I was prepared with an electronics screwdriver set that includes pentalobe bits. I also had a can of compressed air.

I removed the screws and learned that Apple has made the bottom harder to open because now you need a suction cup. I repurposed the suction cup from my car dash cell phone holder.

You have to use the suction cup to lift part of the bottom enough to slide in something to release a few internal latches. I used a library card.

Cleaning the Dust

Once I got my MacBook open, it was obvious that dust was the main cause of my performance issues. Just taking the bottom off caused a cloud of dust to come out.

There was also cat hair from this troublemaker.

Both fans were totally gunked up.

I used compressed air to blow out the dust. I also used a toothpick to get some off the fans that wouldn't come off from blowing alone.

I put the cover back on and haven't had any more thermal performance issues.

Retrospective

It would be nice if macOS could provide some actionable advice when kernel_task is trying to stop the MacBook from overheating. I would have appreciated an OS suggestion to use the right side Thunderbolt 3 ports.

But I also never considered that my laptop fans could require cleaning. In retrospect, that's foolish because I've seen how dirty it can get in desktop PCs, and I know that heat is an inhibitor to computer performance. It's just like running in humid weather. Your body starts shutting down because its main mechanism for getting rid of heat (sweating) stops working because humidity prevents sweat from evaporating.

It would be cool if the machine could detect and alert when the fans are no longer effective, perhaps with a sensor that looks for a lack of airflow at the vents.

To prevent the issue from occurring again, I've set up a recurring task in Todoist to remind me to clean my computers once a year.

Found an error or typo? Feel free to open a pull request on GitHub.

Clearing Mac Storage Space

Danny Guo — Wed, 15 Jul 2020 00:00:00 +0000

I recently needed to clear a large amount of space on my Mac's hard drive in order to install Windows through Boot Camp. After deleting files and applications, I still had large amounts of space taken up in ways that I didn't understand. Here is what I did to fix the problem.

Easy Methods

The obvious thing to start with was deleting known files and applications. In particular, I removed some built-in macOS applications, like GarageBand, that I can always re-download from the Mac App Store.

I also cleared the trash. As a tip, you probably want to set it to automatically delete items after 30 days. I don't understand why that's considered an advanced feature in Finder when it seems like it should be the default setting. It wasn't even an option until Sierra.

Then I went into my Google Backup and Sync (for Google Drive) settings and unsynced some folders that I don't need to be available locally.

Other Storage

After those easy wins, I wanted to get a better idea of what was taking up space. macOS has a built in storage analyzer, but it's pretty basic. The overview gives a rough breakdown.

Clicking manage gives you a more detailed view.

But note that "System" is greyed out! There's no way to drill down into it to see what's actually taking up space.

This was on Mojave. At least Catalina splits it up into "System" and "Other," though you still can't drill down into either.

Disk Space Analysis Programs

I'm familiar with using du -sh in the terminal to show folder sizes, but I thought a GUI program might be better for this task. I looked into third party programs to find out what folders and files were taking up space. The first one I tried was Disk Inventory X.

It was useful but took a while (around 10 minutes) to produce results.

I looked into alternatives. The most helpful resources were this Stack Exchange question and this article.

I tried OmniDiskSweeper. It doesn't have a tree view like Disk Inventory X does, but I didn't find that very useful anyway. OmniDiskSweeper was able to show results as they came, rather than all at once at the end.

It doesn't update in real time, so I did have to re-analyze the disk after deleting files.

Docker

The first thing I noticed was Docker's disk image file. Docker for Mac stores containers and images in that single, large file. I ran docker system prune to delete unnecessary containers and images.

Then I decreased the size limit for the disk image file.

Spotify

The biggest surprise to me was discovering that Spotify can take a large amount of space to cache files. Mine was over 6 GB. Even Firefox's cache was only 2 GB.

Furthermore, there doesn't seem to be a way to control this cache size in Spotify settings, not even buried in some advanced settings. This setting might have existed before, but it doesn't now. Luckily, someone discovered that you can set a limit manually by editing the Spotify preferences file at ~/Library/Application\ Support/Spotify/prefs and adding a line like storage.size=2048.

I set the limit to 2048 MB, restarted Spotify, and confirmed that the cache folder dropped to the expected size, saving me over 4 GB.

$ du -sh ~/Library/Caches/com.spotify.client/Data
6.4G    Data

# restart Spotify

$ du -sh ~/Library/Caches/com.spotify.client/Data
2.0G    Data

Yarn

Yarn seemed to have old cache folders.

$ cd ~/Library/Caches/Yarn && ls v2 v4 v6

$ du -sh v2 v4 v6
  0B    v2
1.2G    v4
1.2G    v6

$ yarn cache dir /Users/dguo/Library/Caches/Yarn/v6

I deleted the v2 and v4 folders. My cache is fairly small, but other people have ended up with gigantic caches.

Git

I found some Git repositories that I didn't need anymore. Deleting these saved around 1 GB in total. Git repos can take up a large amount of space since by default, they contain the entire history of a project. Though in my case, most of the space savings was due to deleting node_modules with each one.

Conclusion

For years, I've used AppCleaner to uninstall applications because I knew most of them leave files behind if you only delete the application itself. This experience reminded me that when I install an application, I generally have no idea where it's going to write files and how large they can get. It would be nice if operating systems could make it easier for users to be more in control of hard drive usage, instead of letting applications treat the hard drive like an unlimited resource.

Found an error or typo? Feel free to open a pull request on GitHub.

Morning Joy - wake up to a happy text every day

Danny Guo — Fri, 01 May 2020 06:59:19 +0000

What I Built

Morning Joy lets you start each day on a good note by texting you an uplifting news story and a picture of something cute.

Following the news can be stressful or even depressing. This is one small way to remember that good things are happening too.

Category Submission

Interesting Integrations

Demo

Link to Code

dguo / morning-joy

Wake up to a happy text every day!

How I Built It

I signed up for Twilio and followed their quickstart guide for Node.js. I bought a phone number and tested sending a text to myself. This was painless. I probably spent more time picking the phone number than I did writing the code.

Next, I needed to find a source for the content that I can count on to be renewed on a daily basis and to be of high quality. I decided to use Reddit's r/aww and r/UpliftingNews subreddits. They are two of the largest subreddits, so I should never get duplicate content. I also decided to grab the first post when sorting by top for today, which means the community has heavily upvoted the post.

I looked into the best way to fetch posts from Reddit. At first I thought I would use their API, but it turns out that Reddit has a convenient feature where you can simply append .json to the end of a subreddit URL to receive the data as JSON: https://www.reddit.com/r/aww.json.

This worked well, but I didn't know how to sort it by top for today rather than hot. I discovered that a query parameter does the trick: https://www.reddit.com/r/aww/top.json?t=day.

I wrote a script to grab the top posts for both subreddits and then text them using Twilio.

Lastly, I needed a way to schedule the script to run every morning. To keep it simple, I converted my script into an AWS Lambda function and set up a CloudWatch Events rule that triggers on a schedule to fire the function every day at 10 a.m. UTC. The schedule expression is: cron(0 10 * * ? *).

This was a relatively simple project, but I'm looking forward to seeing what I get every morning!

Migrating From Authy to Bitwarden for 2FA Codes

Danny Guo — Wed, 25 Mar 2020 00:00:00 +0000

I've used Authy for several years to generate my time-based one-time passwords (TOTP) for two-factor authentication (2FA). For various reasons, I recently migrated to using Bitwarden instead.

Google Authenticator Issues

Many services recommend using Google Authenticator for 2FA. I originally used it before switching to Authy, but I switched for a reason that is still valid today: it doesn't have any sort of backup or syncing functionality.

Check out the reviews to get a sense of how often people get burned by switching to a new phone for whatever reason and realizing they've lost all their codes or need to go through each service one by one and set up 2FA again.

Google Authenticator is also a neglected app. The Android app was last updated on September 27, 2017, and the iOS app was last updated on September 12, 2018. You could argue that these are relatively simple apps that don't need frequent updates, but take a look at what other apps like andOTP and Aegis offer in terms of functionality that Google Authenticator doesn't have, like being able to search for a service instead of having to scroll though the entire list to find it.

Authy Issues

While I have happily used Authy for several years, I also have some issues with it that caused me to look for a replacement.

No Browser Extension

Authy doesn't have a browser extension for Firefox, my primary browser. This is a problem because an extension can offer some protection against phishing, one of the main security weaknesses of using TOTP for 2FA. If the extension fails to find an entry that matches the current domain, that can alert me to a possible phishing attempt.

The Chrome extension also hasn't been updated in two and a half years and will no longer be supported going forward.

No Web Client

Authy doesn't have a web client. While this could be considered a security feature, I'd rather have the option to access my codes through any browser in an emergency. It's a security vs. usability tradeoff that I'm willing to make.

No CLI Client

Authy doesn't have a CLI client. I have some ideas for personal browser automation projects that could be easier to implement with programmatic access to my TOTP codes.

Mac CPU Usage

I use the Mac desktop program, but when it has a code open, the program uses significantly more CPU. Here's the CPU usage when it's just displaying the list of services.

And here's the CPU usage when it's showing the TOTP code.

Since I don't want the program to unnecessarily drain my laptop battery, I try to remember to press the back button after copying the code. There's no option to automatically go back on copy or to just copy the code from the list view without even seeing the code.

Authentication and Recovery

When you create an Authy account, you have to provide a phone number rather than an email address or username. I didn't like this to begin with since I want as few things tied to my phone number as possible, given how often phone numbers get hijacked.

Authy then encourages you to add the app to your other devices and then disable the multi-device feature. This means that your codes will keep working on your existing devices, but to add Authy to a new device, you need access to one of your old ones to temporarily re-enable multi-device and to grant access to the new device. If you don't have access to an old device, you have to go through a 24 hour account recovery process.

However, I want to be able to regain access to my 2FA codes, even if I've lost access to all my devices. For example, I could be in a foreign country without my laptop and then lose my phone. I want to have a good contingency plan for this kind of situation.

Note that Authy doesn't support an account level password. It does support a password for your encrypted backups, but you don't enter that until after you log in.

Authy also doesn't support TOTP codes or U2F security keys for protecting itself. Its sole authentication mechanism (beyond account recovery processes) is access to an old device.

Yubico Authenticator

I considered using my YubiKeys to generate TOTP codes using Yubico Authenticator, but a YubiKey can only store 32 TOTP secrets, and I already have 49 of them since I enable TOTP-based 2FA whenever possible.

Bitwarden

I currently use LastPass to manage my passwords, but I am going to switch to 1Password soon. I decided to use Bitwarden as well but solely for TOTP codes. 1Password can also handle TOTP codes, but I am willing to deal with the hassle of having two password managers to avoid using the same service for both passwords and 2FA.

By using a password manager for TOTP, I get broad cross-platform support with a web client, browser extensions, desktop programs, mobile apps, and even a CLI client. I also get standard authentication mechanisms, including 2FA support.

This does mean that I am treating my TOTP codes more like secondary passwords (something I know) rather than as something I have. Authy's requirement to have access to an old device better fits the latter principle. This is a deliberate choice on my part.

Note that Bitwarden requires a premium account that costs $10 a year in order to generate TOTP codes. A premium account also adds U2F support, which I wanted as well.

Authentication Strategy

U2F support is the last component of my authentication strategy. Going forward, it will be like this: I'll store passwords in 1Password and TOTP secrets in Bitwarden. I'll use separate, high entropy master passwords that will only exist in my head.

1Password requires a secret key in conjunction with the master password in order to log in on a new device. Since I can't memorize it, I plan to store my secret key as a static password on my YubiKeys. This means that if I touch the metal contact for a few seconds, it will type out the secret key for me.

For both services, I'll add all my YubiKeys for 2FA. This means that all I need is one of my YubiKeys (one of which is on my keychain) and the master passwords in my head to regain full access to all of my accounts.

However, I can't guarantee that I'll be able to use my YubiKey on every device. For example, Bitwarden doesn't support U2F in its mobile apps. I would also be paranoid about feeling like I need two YubiKeys when I travel in case I lose one.

My plan to deal with these issues is to also set up TOTP-based 2FA for both 1Password and Bitwarden. I'll print those TOTP secrets, along with the 1Password secret key, on a small card and laminate it. I can make multiple copies to put in my wallet and my bag. Sometimes being overly prepared is fun in itself, even though it might be overkill.

Migration

To migrate to Bitwarden, I went through my Authy list one by one. In theory, I'd be able to just copy the TOTP secret to Bitwarden, but Authy doesn't expose the secret.

For each account, I logged in and reset 2FA to add the secret to Bitwarden. Then I deleted the account from Authy. Authy marks it for deletion and then waits 48 hours before actually deleting it in case you made a mistake.

I did have trouble with adding some services, such as Algolia and npm, that only show the QR code and don't have an option to display the TOTP secret. The QR codes encode URIs that look like this, as documented in the Google Authenticator wiki:

otpauth://totp/Example:alice@google.com?secret=JBSWY3DPEHPK3PXP&issuer=Example

I tried using my phone camera's built-in QR scanner, but I couldn't see the full URI and opening it would open Authy, with no other option. I used Google Lens instead to grab the secret. In retrospect, I was only having trouble because I was adding the services to Bitwarden through the browser extension. I should have installed the mobile app from the beginning and used that because it has an option to scan QR codes.

I also had trouble with adding Twitch, which has a specific integration with Authy instead of providing a generic QR code. To get around the issue, I followed this guide. You can use the deprecated Authy Chrome app to retrieve the TOTP secrets and configurations. This method entails using Chrome's developer tools to execute custom code to print the information.

This revealed that Twitch uses 7 digit codes instead of the standard 6 and 10 second intervals instead of the standard 30.

otpauth://totp/Twitch?secret=XXXXXXXX&digits=7&period=10

At this point, I thought I hit a Bitwarden limitation because I mistakenly assumed that the extension would only take the TOTP secret in the authenticator key field.

However, I discovered that Bitwarden supports putting the full URI with configuration into that field. I tested it and was able to log in to Twitch using the code generated by Bitwarden.

Conclusion

Migrating to Bitwarden took me about a full day, but I'm happy with the result. I've been using the Bitwarden browser extension to log in to accounts for the past week, and it is much nicer than using the Authy desktop program. Next up is migrating from LastPass to 1Password.

Found an error or typo? Feel free to open a pull request on GitHub.

How to Add Copy to Clipboard Buttons to Code Blocks in Hugo

Danny Guo — Fri, 22 Mar 2019 00:00:00 +0000

A small quality of life improvement for programming-related websites is to add copy to clipboard buttons to code blocks. When a visitor wants to copy a code example or a shell command, it's nice to be able to just hit a button rather than manually select the text, right click, and press copy.

I use Hugo to build my personal website. While Hugo has built-in support for syntax highlighting, it doesn't support copy buttons. Here is how I added the feature to my website. The end result looks like this:

Adding the buttons

I inspected the source of a page with code blocks and found that Hugo generates each block with markup like this:

<div class="highlight">
    <pre>
        <code>...</code>
    </pre>
</div>

Code blocks without syntax highlighting have the same structure but without the surrounding highlight div. To account for both cases, I selected for code elements that are children of pre elements.

document.querySelectorAll('pre > code').forEach(function (codeBlock) {
    var button = document.createElement('button');
    button.className = 'copy-code-button';
    button.type = 'button';
    button.innerText = 'Copy';

    var pre = codeBlock.parentNode;
    if (pre.parentNode.classList.contains('highlight')) {
        var highlight = pre.parentNode;
        highlight.parentNode.insertBefore(button, highlight);
    } else {
        pre.parentNode.insertBefore(button, pre);
    }
});

For many implementations of copy code buttons that I've seen, the button is located in the top right or bottom right corner of the code block. However, I've noticed that the button can cover up some of the code if the line is too long, especially on mobile. To avoid this possibility, I placed each button before the entire code block.

Some implementations only show the button when the user hovers over the code block, but for discoverability, I left the buttons always visible.

For styling the buttons, I used this CSS:

.copy-code-button {
    color: #272822;
    background-color: #FFF;
    border-color: #272822;
    border: 2px solid;
    border-radius: 3px 3px 0px 0px;

    /* right-align */
    display: block;
    margin-left: auto;
    margin-right: 0;

    margin-bottom: -2px;
    padding: 3px 8px;
    font-size: 0.8em;
}

.copy-code-button:hover {
    cursor: pointer;
    background-color: #F2F2F2;
}

.copy-code-button:focus {
    /* Avoid an ugly focus outline on click in Chrome,
       but darken the button for accessibility.
       See https://stackoverflow.com/a/25298082/1481479 */
    background-color: #E6E6E6;
    outline: 0;
}

.copy-code-button:active {
    background-color: #D9D9D9;
}

.highlight pre {
    /* Avoid pushing up the copy buttons. */
    margin: 0;
}

Interacting with the clipboard

Next, I investigated how to copy to the clipboard using JavaScript. The most popular library for doing so is clipboard.js, but I wanted to avoid bringing in a dependency if possible.

One way is to use execCommand with document.execCommand('copy'), which copies the current text selection. Under the hood, clipboard.js uses this method.

However, there is a newer approach, the Clipboard API. It has several advantages: it's asynchronous, takes arbitrary text/data (so it doesn't have to already exist on the page), and has a better story for dealing with permissions. Chrome, Firefox, and Opera support it already. For other browsers, there is a polyfill.

I put the code in a function and added click handlers. I used innerText to get the code to be copied. After the copy operation, the button displays either an error message or a success message that lasts for two seconds.

function addCopyButtons(clipboard) {
    document.querySelectorAll('pre > code').forEach(function (codeBlock) {
        var button = document.createElement('button');
        button.className = 'copy-code-button';
        button.type = 'button';
        button.innerText = 'Copy';

        button.addEventListener('click', function () {
            clipboard.writeText(codeBlock.innerText).then(function () {
                /* Chrome doesn't seem to blur automatically,
                   leaving the button in a focused state. */
                button.blur();

                button.innerText = 'Copied!';

                setTimeout(function () {
                    button.innerText = 'Copy';
                }, 2000);
            }, function (error) {
                button.innerText = 'Error';
            });
        });

        var pre = codeBlock.parentNode;
        if (pre.parentNode.classList.contains('highlight')) {
            var highlight = pre.parentNode;
            highlight.parentNode.insertBefore(button, highlight);
        } else {
            pre.parentNode.insertBefore(button, pre);
        }
    });
}

Next, I added a check for whether or not the browser supports the Clipboard API. If not, the script loads the polyfill from CDNJS.

if (navigator && navigator.clipboard) {
    addCopyButtons(navigator.clipboard);
} else {
    var script = document.createElement('script');
    script.src = 'https://cdnjs.cloudflare.com/ajax/libs/clipboard-polyfill/2.7.0/clipboard-polyfill.promise.js';
    script.integrity = 'sha256-waClS2re9NUbXRsryKoof+F9qc1gjjIhc2eT7ZbIv94=';
    script.crossOrigin = 'anonymous';
    script.onload = function() {
        addCopyButtons(clipboard);
    };

    document.body.appendChild(script);
}

After the Clipboard API becomes ubiquitous, I'll remove the polyfill code.

Smart loading with Hugo

After I got the functionality to work, I thought about how to include the script. I had three options. The first was to indiscriminately include it on every page. The script is small, but for optimization, I wanted to only include it when it's actually needed, saving a bit of bandwidth and a network request (or two, if the polyfill is needed).

The second option was to use a custom Hugo front matter variable. With this method, I'd set a flag on every post that has a code block. The template could then check for this flag. However, this approach involves manual work and runs the risk of me forgetting to do it.

The third option was to find a way to use Hugo to figure out which pages have at least one code block. A regex seemed like the way to go. I used Hugo's findRE function to determine if the HTML seems to contain a pre element.

{{ if (findRE "<pre" .Content 1) }}
    <script src="/js/copy-code-button.js"></script>
{{ end }}

I passed it a limit parameter of 1 because I only care if the page has a code block or not, not the total number of code blocks.

Keep in mind that this script should be loaded after the page content, preferably at the end of the body so that it doesn't block rendering. Otherwise, the selector might run before the code blocks actually exist.

Non-Hugo websites

This solution should easily work for non-Hugo websites as well. The only part of the script that is specific to Hugo is the pre > code selector. Modifying the selector and possibly where the button is inserted should be all that is needed.

CodeCopy

CodeCopy is a browser extension for Chrome and Firefox that adds copy buttons to code blocks on many websites that are likely to have them, such as GitHub and Stack Overflow. It's made by the same person behind clipboard.js.

Found an error or typo? Feel free to open a pull request on GitHub.

Animated Multiline Link Underlines with CSS

Danny Guo — Thu, 20 Dec 2018 21:33:06 +0000

One benefit of building my personal website from scratch instead of using a theme made by someone else is that I can start from the browser's defaults and gradually add my own flourishes. I strive to keep my site lean, but making it personal is also kind of the point. There is a spectrum of gratuitous touches between the spartan pages of Hacker News and Craigslist on one end and the sensory overload of old MySpace on the other.

I ran across a site that had fancy, animated underlines for links, and I wanted to add a similar effect to my site. It was important to me to use a pure CSS solution. For something as frivolous as this, I didn't want to add JavaScript that has a risk of causing a performance issue or frustrating behavior (see scroll hijacking).

Here's what the effect looks like now.

Implementation

The topic of drawing lines under text on the web can be surprisingly complicated, depending on how far you are willing to stray from text-decoration: underline and which details (like clearing descenders) you care about.

I investigated a few approaches:

Both of them essentially remove the default text-decoration and add a simulated border using pseudo-elements. The border is then animated by CSS transitions. Unfortunately, these solutions have one drawback: they don't work properly if the link spans more than one line. The underline only appears on the first line.

I eventually found a CodePen by Shaw that doesn't have this flaw. I modified the CSS until I got to a solution that looks good to me.

Here is the relevant code. You can use this repl to play around with it.

a {
    text-decoration: none;
    background-image: linear-gradient(currentColor, currentColor);
    background-position: 0% 100%;
    background-repeat: no-repeat;
    background-size: 0% 2px;
    transition: background-size .3s; 
}

a:hover, a:focus {
    background-size: 100% 2px; 
}

Let's go through this approach part by part. First, we turn off the default text-decoration for links.

We want to use background-image because it can span multiple lines. While we could supply an actual image, we only want to draw a line, so we use linear-gradient, which generates an image for us. Normally, it's used to create a gradient between two different colors, but I want the underline to just be the same color as the link, so I use currentColor for both the beginning and the end of the gradient. currentColor tells the browser to use the element's computed color property.

We use background-position to place the image in the bottom left corner. 0% sets the horizontal position, and 100% sets the vertical position.

We turn off background-repeat to prevent it from creating multiple instances of the image to fill the entire background of the link.

We use background-size to make the image zero pixels wide and two pixels tall. It has zero width because the underline should only appear on hover.

We set a transition on background-size, so any change to the property will take 0.3 seconds to complete.

On link hover, we change the width of the image to 100%, creating a full underline, and transition takes care of the animation. As nickels55 suggests, we also want the effect to happen on focus for those who use keyboard navigation. Thanks to nickels55 for keeping accessibility in mind.

And that's it! I was very happy with how small the commit ended up being. If you'd like to add something similar to your site, feel free to take this implementation, or check out some other animated underline effects for inspiration.

Originally published at www.dannyguo.com.

Found an error or typo? Feel free to open a pull request on GitHub.

Using Mailgun for a Free Custom Domain Email Address

Danny Guo — Fri, 07 Dec 2018 23:37:37 +0000

I've been using Gmail for my primary email address for over 13 years. I should thank my sister for making me use my name and not my now embarrassing AIM username. Here's a look at the welcome email back in 2005. Gmail wasn't even a year old yet.

Gmail has served me well since then, but mostly for fun (and partly because of vanity), I recently set up an email address on my own domain. I ended up going with a free solution that involves Mailgun and is good enough for my purposes. I wanted to still use my Gmail account to actually receive and send the custom domain emails. It isn't important to me right now to actually have a separate inbox or to stop using Google for email.

Options

Several years ago, a common choice was Google Apps (now known as G Suite) because it was free for up to five users. G Suite got rid of the free edition and costs $5 per month now, so I investigated alternatives.

I've read about Zoho Mail as a free alternative. Unfortunately, it stopped offering POP and IMAP access for its free plan earlier this year, so the only way to access my email would be through Zoho's website or apps. It would cost $3 per month to add POP and IMAP support. I don't fault Zoho for making this change, but I would prefer to access all my email through my Gmail account. I have also read some anecdotes that Zoho Mail can be slow in sending and receiving emails.

Next, I looked at FastMail and ProtonMail, which are frequently cited by people who have chosen to avoid Google services. FastMail costs $5 per month or $50 per year for a plan that includes custom domain support. ProtonMail requires a paid plan for the feature, and the cheapest paid plan is $5 per month or $48 per year.

Another option was to use Google Domains. It includes a free email forwarding feature that allows for sending an email from the alias as well. However, the feature is only available when you use Google's name servers. My site runs on Netlify, and I use their DNS service.

I also looked at mailboxes from other domain registrars. Gandi includes two mailboxes with every domain, but Gandi's domain prices also seem more expensive than elsewhere, especially when considering that Cloudflare's at-cost domain registrar service is scheduled to be available soon.

Namecheap has very affordable email hosting that only costs $9.88 a year. I was ready to set it up when I stumbled upon one last solution, deep in search results for "custom domain email." Some people use Mailgun.

Mailgun

Essentially, incoming emails go to Mailgun. With a routing rule, I can forward those emails to my Gmail, and with some additional configuration, I can also send emails from my Gmail but have them appear as if they are sent from my custom domain.

Mailgun's free tier is way more than enough. It allows for 10,000 emails per month. That includes both incoming and outgoing emails, but for a personal email account, I'll never even come close to that limit. Even if I do manage to go over, it only costs $0.50 for every additional thousand emails.

The plan does restrict you from sending to email addresses that you haven't explicitly added to an "authorized recipients" list. However, Mailgun lifts this restriction if you provide a payment method, which puts you on the "concept" plan. I was happy to do so.

Do note that Mailgun does not officially recommend using the service for personal email, but they do not forbid it. They recommend using hosted email services like G Suite instead, which I would definitely do for something like a business.

Instructions

Here are the step by step instructions. Sign up for a Mailgun account, and provide a payment method unless you're okay with the sending restriction.

Go the "Domains" tab to add a new domain. The page recommends using a subdomain, like "mail.yourdomain.com" instead of just "yourdomain.com". The reason for this recommendation is that in general, you don't want to risk hurting your root domain's email sender reputation. Companies will sometimes use different domains or subdomains for sending transactional versus marketing emails.

For the purpose of a personal email address, use the root domain if you want to receive email there (ben@yourdomain.com) rather than at a subdomain (ben@mail.yourdomain.com). The risk of harming your sender reputation is much lower because you presumably aren't going to use your personal email to email tons of strangers.

Next, go to wherever you manage your website's DNS settings (like your domain registrar), and add the DNS records provided by Mailgun. The MX records are for setting Mailgun's servers as the recipient for emails. The TXT records are for setting up SPF and DKIM, which are two ways for recipients to verify an email's authenticity. You don't need to add the CNAME record unless you want to track what your recipients do with your emails, like opening them and clicking links inside them.

It might take a while for the DNS changes to propogate, though mine only took about a minute. Tell Mailgun to check the DNS records. Once they are verified, you can go to the "Routes" tab to create a routing rule.

Change the expression type to "Match Recipient," and set the custom domain address that you want to use. Then check the "Forward" action, and set the address where you actually want to receive the emails (like a Gmail account). You could set up more sophisticated rules, or even add additional addresses that all forward to the same email address.

At this point, receiving emails should work. Go ahead, and try it out. I also created a Gmail label, and set up a filter so that all emails from my custom domain get tagged with it, making it easy for me to view only those emails if I want to.

To make sending work, you'll need some additional information from Mailgun. Go back to the domains tab, and open your domain. You'll need the values for "SMTP Hostname," "Default SMTP Login," and "Default Password."

Go to Gmail settings, and open the "Accounts and Import" tab. In the "Send mail as" section, add the custom domain email address. I unchecked the "Treat as an alias" option because I want to automatically reply with whichever address is the intended recipient. I don't want to have to manually change the sender. Normally, unchecking the option means that emails sent to the other address won't show up in Gmail, but in this case, Mailgun is handling that part.

Go to the next step, and fill in the info from Mailgun. The Mailgun "SMTP Hostname" corresponds to the Gmail "SMTP Server" field, and the "Default SMTP Login"" corresponds to the "Username" field. Leave the SMTP port as 587, and make sure to fill in the password.

Press the "Add Account" button, and you should get a confirmation email from Gmail that contains a link you can use to verify the request. At this point, sending emails should work too! When you compose an email, you can click on the "From" field to change who you are sending as.

Try it out. Your recipient should receive the email, and the email details should have both the "mailed-by" and "signed-by" fields set to your custom domain. From their perspective, they don't know that you actually used Gmail to send the email.

One last thing you can do is to set a monthly sending limit with your Mailgun account. The option is in your account settings. Look for the "Custom Message Limit" field. Just in case someone manages to get access to your Gmail and tries to use your custom domain address to send email, Mailgun will stop processing emails once the limit is hit.

Conclusion

If Mailgun does change in a way that makes it an unreasonable solution, I will probably switch to a Namecheap or ProtonMail mailbox. Until then, I'm happy to have found a free solution for a custom domain email address.

Originally published at www.dannyguo.com.

Found an error or typo? Feel free to open a pull request on GitHub.

I Published My AWS Secret Key to GitHub

Danny Guo — Wed, 15 Aug 2018 00:00:00 +0000

One of my earliest programming projects was a basic news headlines aggregator that would display the top three headlines for each news source. I called it The Daily Lore. The original site was hosted on AWS S3 as a static website. At the time, I didn't know how to make a dynamic, server-generated website, so my hacky solution for updating the headlines was to run a Python script that would generate a JavaScript file with the headlines hardcoded inside. The script then updated the JS file in the S3 bucket. To automate the updates, I ran the script on a micro EC2 instance in a cron job. Crucially, that means the script needed an AWS secret key.

Eventually, AWS terminated my instance for some reason, and I abandoned the website for a while. I revived it a few years later with a new repo. Last month, I was cleaning up my Google Drive when I came across the old source code. I had stored the .git folder as well, so I decided to push the repo up to GitHub just for fun.

A few minutes later, I got an email:

Amazon Web Services has opened case ******** on your behalf.

The details of the case are as follows:

Case ID: ********

Subject: Your AWS account ******** is compromised

Severity: Urgent

Correspondence: Dear AWS Customer,

Your AWS Account is compromised! Please review the following notice and take
immediate action to secure your account.

Your security is important to us. We have become aware that the AWS Access
Key ******** along with the corresponding Secret Key is publicly available
online at ********.

This poses a security risk to your account and other users, could lead to
excessive charges from unauthorized activity or abuse, and violates the AWS
Customer Agreement.

Please delete the exposed credentials from your AWS account by using the
instructions below and take steps to prevent any new credentials from being
published in this manner again. Unfortunately, deleting the keys from the
public website and/or disabling them is NOT sufficient to secure your
account.

To additionally protect your account from excessive charges, we have
temporarily limited your ability to create some AWS resources. Please note
that this does not make your account secure, it just partially limits the
unauthorized usage for which you could be charged.

Detailed instructions are included below for your convenience.

Immediate actions

First, I let out a loud groan. I immediately knew the consequences of what I had done. I've read plenty of stories of nefarious people scraping AWS keys and spinning up tons of EC2 instances for purposes like cryptocurrency mining.

I deleted the GitHub repo, but like the email cautions, I knew that wasn't enough. Once a secret is compromised once, it's compromised forever. I went into the AWS console and invalidated the key. Luckily, this was on my personal AWS account, and nothing else depended on that key. I got another AWS email a few minutes later confirming that the compromised key had been deleted.

Then I went through each AWS region to make sure nothing suspicious was present, like newly created servers or IAM users.

Cleaning up and republishing

I still wanted to publish the repo, but I also wanted to scrub the secrets. Even though they were no longer usable, I didn't want anyone to see them in the future and get the impression that committing secrets is okay. It wouldn't be enough to just remove them in a new commit because they would still be in the Git history. This part was the silver lining because I knew about the BFG (I assume the name is a reference to the weapon in Doom, but I'm not sure) but had never gotten a chance to use it before. It made it very easy to remove the secrets from the entire history.

I did a more thorough inspection and found other things in the repo that shouldn't have ever been committed. There were .pyc files. There was a PDF explaining how SVGs work. There was an Excel spreadsheet containing passwords for services that I used for the site. At the time, I wasn't using a password manager. Fortunately, I had changed all the passwords when I did start using a password manager, so I didn't need to do so again. Lastly, there was even a SSH private key for logging into the EC2 instance, and a script for doing so that contained the host.

I used the BFG to remove all of these files. I double checked to make sure it worked as expected, and then I re-pushed the repo to GitHub.

Retrospective

I should have done a manual inspection of the files before pushing. I also should have used a tool that is designed to find secrets in Git repos. There are several of them:

I should have distrusted my past self. While it's secondhand nature to me now to use environment variables for secrets, to think about how they can be leaked, and to be aware of which files should and shouldn't be committed to Git, I forgot that at one point, I didn't know any of that. Lesson learned.

Found an error or typo? Feel free to open a pull request on GitHub.