DEV Community: Edwin Mak

Skip turbo broadcasting to speed up seeding process in Rails

Edwin Mak — Thu, 24 Nov 2022 11:08:47 +0000

Introduction

We discovered that our seeding process for our rails application was getting very slow by turbo broadcasting via callbacks triggered by saving models. Although this works as intended, we certainly don't need to broadcast turbo frames in the seeding process and just serves to slow down the seeding process without any benefits.

Solution

We were able to speed up the seeding process by 3x (~140s to ~40s) by disabling turbo broadcasting in seeding. You can disable them too by inserting this code snippet before your seeding process:

class Turbo::StreamsChannel
  [:broadcast_append_to, :broadcast_prepend_to, :broadcast_replace_to, :broadcast_remove_to].each do |method|
    define_singleton_method(method) do |*args|
      return nil
    end
  end
end

# SEEDING PROCESS BEGINS

Conclusion

We were able to improve the developer experience by making the
seeding process quicker with a short snippet of code. Shaving off about a minute of seeding seems insignificant, but it adds up!

Hope this helps!

How to find and identify a great mentor for me?

Edwin Mak — Wed, 29 Jan 2020 20:39:55 +0000

Hello DEV.to,

I am currently looking for a mentor to guide & facilitate my growth as a technical professional & software engineer. What characteristics should I be looking for in a mentor? And where would be the best place to find this person?

Most likely this is a very specific ask with different answers depending on my needs and preferences. In summary, I am a relatively new (less than 1 year) Lead Engineer on a tiny team with about 5 years of Ruby On Rails working experience. There are topics related to legacy code and hiring that I would like to talk with a more experienced person about.

I would love to hear any advice on finding this person! Open to talking with any mentors that are reading this and is looking for an eager learner :)

How to safely drop a deprecated database table

Edwin Mak — Sat, 11 Jan 2020 16:24:16 +0000

Stop! Do not immediately drop a table because it appears unused or deprecated!

I have a safer process for dropping deprecated tables that is less dangerous and mitigates issues! Dropping a table can be very dangerous as you could lose all the historical data of that table if you don't have a backup. Also, you could be shooting yourself in the foot; if the table is still being used somewhere in your application, this undoubtedly introduces bugs. The later is more likely to happen if your application is not well tested. And most codebases that I've encountered are not well tested, so this process is likely applicable to you.

Developers, including myself occasionally, have a terrible tendency to trust their memory and expertise of the application's internal workings. Just because you have a firm grasp of the application internals, you shouldn't just pull the trigger and drop the table. We are all human, we can forget and make mistakes. Do not rely on your memory instead rely on a process.

Here is the process that I follow and highly recommend when dropping a deprecated table.

The Steps

Step 1 - Determine if this table is indeed 'deprecated' and no other team is using it.

You must be sure that the table in question is not being used anymore by anyone! Products like [Looker] have given non-technical people access to your database tables. The external team now has the power to analyze data and act on them. They might rely on it to perform their operations. You should reach out to other teams that you believe could be using it and get their consent on the deprecation process. If someone is relying on it, then stop and rethink the deprecation. Otherwise, let's move onto the next step.

Step 2 - Change the database table's name.

Rename the table's name to deprecated_{date}_tablename to see if you missed any issues referencing to that table. Bug reports (I recommend having a bug monitoring system in place first) will inform you of these issues. No worries. We can rollback to change the name back and handle the damage control. The damage incurred would be significantly less severe than actually dropping the table. Rollback and then start again from step (1).

Step 3 - Let the renamed deprecated table 'marinate' in production.

You'll need to let the renamed deprecated table stay on production for enough time to indicate that it is safe to move on. Sometimes you'll have code that references to the table that doesn't execute very often. In this step, you'll need to decide how long is long enough. You can think about your applications usage profile to better help you decide. For instance, if you are a point of sales system that generates weekly reports to your clients then you might want to wait a week. This time can vary between different use cases, so I'd suggest using your best judgment here.

Step 4 - Drop the table! Boom!

Now that we've gone through the precautionary steps 1, 2, and 3. You can drop that renamed table! Let's help that database shed some weight~

Summary

There you have it! These are the steps that I take when I am deprecating a database table (BTW also works for columns too). This process has saved me from headaches on numerous occasions. I had halted deprecation of a table after I discovered it was being used heavily by another team. In another case, I found that I didn't remove all references to the table and rolled back smoothly so that I could address them.

The best thing about having steps and a process is that you can rely on and trust in the process. It is too much for a single developer to accurately memorize every detail of a niche part of your much broader application.

Even if you choose not to use these steps, I still urge you to take extra precautions when dropping a database table for the sake of yourself and the other teams that rely on the database.

Acknowledgement

Thank you Adam Milligan for teaching me this approach.
Featured image of the traffic light was downloaded from here.

A hilarious game to illustrate why UX is important!

Edwin Mak — Mon, 08 Jul 2019 16:56:40 +0000

As the title implies, this is the game developed by the cool people of Bagaar to show how UX is super important!

You can play the game here:
https://userinyerface.com/

Disclaimer:
I'am not responsible for any thrown phones or smashed keyboards.

Sprinkling StimulusJS - Light JS Without Heavy Choices.

Edwin Mak — Sun, 07 Jul 2019 17:39:55 +0000

Introduction

In today's state of web applications, it can be hard to choose the right tool to imbue your web pages with bits of interactivity. There is an immense number of options that you can take, but unfortunately, many of them require you to commit to a comprehensive javascript framework. This decision to commit I believe should not be taken lightly especially if you are making this choice for your team/company/business.

The way I often handle these decisions is to not avoid making them at all. Decision fatigue and the stress it brings are real.

In an evasive attitude of risk, I discovered StimulusJS as a possible option that fits some of the use cases. In this article, I share my experience with it in creating an interactive password input form with requirements being shown as being satisfied or not on user typing and my thoughts on it.

Sprinkle Stimulus

As the authors mentioned, the goal of Stimulus is to sprinkle pages with JS to make them sparkle. If you primarily have server-side rendered HTML pages and you want to imbue some interactivity on them, then it might be a good fit for your project. Otherwise, you might want to find another tool.

The core concepts specifically mention that it is mainly meant to bind onto existing HTML and manipulating them. It is not concerned with creating HTML to render like React does.

For instance:

// In Stimulus
<div data-controller="clipboard">
  PIN: <input data-target="clipboard.source" type="text" value="1234" readonly>
  <button data-action="clipboard#copy">Copy to Clipboard</button>
</div>

// In React
<pin-input></pin-input>

In the Stimulus approach, you'd see all the HTML up front and might avoid having to dig into another file to see what HTML would be. However, this could mean your HTML files can get larger and run the risk of being unmanageable.

StimulusJS Is Pretty Easy To Learn

As a person who spends most of their time working in the backend, I found that the documentation was clear and easy to digest. I was able to get the main premise of it and enough information to use it fairly quickly. I spent about an hour (or so) with the documentation and was able to be productive with it.

Building An Interactive Password Input Field With Requirements

In my pursuit of learning, I created a basic password input field that shows the requirements beneath it in an interactive manner. The requirements beneath are toggled to be satisfied or not satisfied upon user input. You can see the source code here and test it live here.

A demo gif of it -

Here is are snippets derived from the code that demonstrates it working together (with some code removed for brevity):

import { Controller } from "stimulus";

class PasswordInputController extends Controller {
  static targets = [
    "lowercaseLetter"
    // Removed other targets for brevity.
  ]

  connect() {
    this.validatePassword();
  }

  onInput(event) {
    this.password = event.target.value;
  }

  set password(value) {
    this.data.set("password", value);
    this.validatePassword();
  }

  get password() {
    if (!this.data.has("password")) {
      this.password = "";
    }

    return this.data.get("password")
  }

  validatePassword() {

    // Validate lowercase letters
    let lowerCaseLetters = /[a-z]/g;
    if (this.password.match(lowerCaseLetters)) {
      this.lowercaseLetterTarget.classList.toggle('invalid', false);
    } else {
      this.lowercaseLetterTarget.classList.toggle('invalid', true);
    }

    // Removed code on other validations of brevity
  }

}

export { PasswordInputController }

<div data-controller="password-input">
  <input class="input" type="password" data-action="input->password-input#onInput"></input>

  <li data-target="password-input.lowercaseLetter" class="password-requirements__requirement">
    At least one lowercase letter
  </li>
  <!-- Remove Code For Brevity -->
</div>

The PasswordInputController defines the logic that runs in relation to the HTML that it will be bound to. The Controller class provided by Stimulus, gives us the ability to have sections of logic to run upon binding via connect() and on user input defined by onInput() and the data-action="input->password-input#onInput" attribute on the input element. A validatePassword() function is defined to add/remove the invalid CSS class of the corresponding li elements via the data-target attribute. This function is called initially by connect() and also within the setter of password.

Through this, I was able to produce a functional password input field.

Testing?

I found that testing the controllers made via Stimulus was not as simple as I would have liked.

I initially ran into some issues trying to get the stimulus controllers to run in due to missing MutationObserver. This was remedied by a post by Hrvoje Šimić.

Unfortunately, there are not many resources related to how to testing. However, a issue had been opened recently to request added documentation on the official webpage.

Admittedly, my struggles with getting the tests setup might be rooted in my inexperience.

Conclusion

Stimulus is a great tool that can be used to make your HTML pages "sparkle" with interactivity without having to commit to a framework. It was fairly easy and approachable to learn how to utilize the Stimulus.

If you find yourself needing to add a little bit of interactivity onto your predominantly server-side HTML rendered application, you might find salvation in choosing to use Stimulus.

Explain Selenium & Webdrivers Like I'm Five

Edwin Mak — Sun, 12 May 2019 16:42:52 +0000

Hey there!

I've been interested lately in applying end-to-end tests to my web application. I've seen several options available but I don't really understand the differences or how they work.

For instances, how does a remote selenium server work in practice? How does the remote selenium server have access to say a server running on my local machine (if thats how it works?)

This area is very new to me and am eager to learn more :)

Thanks!

Enhance your PostgreSQL practice on MacOSX with Postico

Edwin Mak — Tue, 07 May 2019 14:30:05 +0000

Introduction

Disclaimer - Postico is only supported by MacOSX.

I've noticed on numerous occasions of developers that might be giving themselves a more difficult time practicing or playing with their PostgresSQL database using the default interactive terminal. Admittedly, I was also one of these developers until my boss at the time offered me an alternative. He suggested using Postico, a PostgresSQL client, and I've been happily using it since. Of course, there are several PostgresSQL clients available with their various advantages & disadvantages.

In terms of simplicity, ease of use, minimalism, I believe Postico is a great fit for beginners or a basic user that just wants to wipe up some queries quick. Here are some of my reasons:

Note - I am not financial tied with the success of Postico. I am just a web developer who thinks this can help others :).

Free Version Is Useful

The free version of Postico provides enough utility to be useful and is not super annoying. There are some products out on the market that makes their free edition very annoying to use as a sales tactic. The free edition of Postico does come with limitations but aren't those that would make you want to pull your hair out.

Here are the limitations as per their FAQ:

I feel like these limitations aren't the most annoying for most use-cases. Plus, you can use the free edition for as long as you want!

Please purchase their license to support them if you have the means to :)

Their Minimalist UI

Their biggest selling point is the minimalist UI in their product. On first load, you are presented with a few options that feel intuitive and easy to navigate.

Adding and saving configurations for accessing local or deployed databases is a breeze.

The UI for the editor & query results is clear and intuitive.

Saving & Loading Queries

Fairly self-explanatory, but the ability to easily save and load your queries is a must have in these client tools. Nothing worst then building a super awesome query and then losing it!

Executing select highlighted portions of your SQL + 'Execute Statement' hotkey

There have been times I find myself wanting to fiddle with multiple queries side-by-side. I found that having multiple tabs and windows can be distracting. Hence, being able to write out multiple queries and running them selectively has been a total game changer!

Heres a quick gif of this feature in action (Press CMD + Return to trigger hotkey to "Execute Statement"):

Conclusion

I am obviously a big fan of using Postico. But that doesn't mean that this is the right tool for everybody forever. You may find that there are other tools that resonate with your needs more and you should reach for it! Wither you decide to use Postico or not, I hope that you've been encouraged to find tools that can improve your experience as a developer working with SQL.

Our jobs are difficult enough and filled with complexity, a little bit of convenience here and there goes a long way.

Thanks!

Convert YAML column data to JSONB in Rails and Postgres

Edwin Mak — Sun, 05 May 2019 00:56:45 +0000

Introduction

My team had wanted to take advantage of the incredibly useful JSON features provided by Postgres 9.4+ in working with data that our rails application was storing as YAML in a text column. We had installed a very useful gem called audited that was responsible for keeping track of "when" & "what" changed in our ActiveRecord records. The "what" data was, unfortunately, being stored as YAML in a text column and makes querying this data more difficult than it needs to be. Here is a sample of that "what" data:

---
pilot:
- false
- true

And this is what I want it to be like:

{
  "pilot": [false, true]
}

Notably, the authors of the audited gem included the option to install it such that this data is stored in JSONB (default is YAML).

Lately, we have been interested in building more advanced queries to search by the content of this "what" data. As a result, we aimed to create a migration that can transform the data safely back and forth from YAML to JSONB (reversible for good measure).

The Result

Here is a migration file that we ended up generating that converts the audited_changes column (that "what" data) on the audits table from YAML to JSONB. You can also utilize this snippet of code to transform your YAML column to JSONB.

Disclaimer - Migrate at your own risk!

class ChangeAuditedChangesFromYamlToJson < ActiveRecord::Migration
  def up
    updated_records = fetch_audit_records.map do |r|
      {
        id: r['id'],
        audited_changes: escape_sql(JSON.dump(YAML.safe_load(r['audited_changes'])))
      }
    end

    remove_column :audits, :audited_changes
    add_column :audits, :audited_changes, :jsonb

    Transaction.transaction do
      updated_records.each do |ur|
        ActiveRecord::Base.connection.execute(
          """
            UPDATE audits 
            SET audited_changes = '#{ur[:audited_changes]}'::jsonb 
            WHERE id = #{ur[:id]}
          """
        )
      end
    end
  end

  def down
    updated_records = fetch_audit_records.map do |r|
      {
        id: r['id'],
        audited_changes: escape_sql(YAML.dump(JSON.parse(r['audited_changes'])))
      }
    end

    remove_column :audits, :audited_changes
    add_column :audits, :audited_changes, :text

    Transaction.transaction do
      updated_records.each do |ur|
        ActiveRecord::Base.connection.execute(
          """
            UPDATE audits 
            SET audited_changes = '#{ur[:audited_changes]}'::text 
            WHERE id = #{ur[:id]}
          """
        )
      end
    end
  end

  private

  #
  # Returns the id & audit_changes column of the audits table.
  # @return [PG::Result]
  def fetch_audit_records
    ActiveRecord::Base.connection.execute('SELECT id, audited_changes FROM audits')
  end

  #
  # Returns the SQL escaped version of the string provided 
  # For example, a "'" needs to be escaped by doubling it.
  # @param [String] the string to be converted
  # @return [String] the escaped version of the string provided
  def escape_sql(string)
    return string.gsub("'","''")
  end

end

Conclusion

It took some trial and error and navigating through some 'gotchas' to arrive at our resultant migration. For instance, we had to escape single-quote characters by adding another single-quote next to every instance (See the escape_sql method in the code snippet). Nonetheless, we are happy with the results and excited to start building queries with those super awesome JSON features given to us by Postgres.

Hope this helps someone else =). Happy coding!

Seeking suggestions for leveling up web design skills

Edwin Mak — Mon, 29 Apr 2019 15:03:57 +0000

Hello DEV community,

I am seeking to level up my abilities as a designer to be able to boost my confidence building visually pleasing (Sorry I know this is vague) websites & UI.

I believe I exercise a good amount of HTML & CSS as I can take a design given to me and create it in code. I'd like to be able to start designing web UIs that I can myself translate into code. I was wondering if there are courses, resources, tips on how I can get better..

Thanks a ton!

Explain Required Sign In Forms For Wifi Access Like I'm Five

Edwin Mak — Fri, 26 Apr 2019 14:56:14 +0000

Hello there!

I've always wondered how some wifi connections require you to sign in or fill out a form on a page before you can freely access the internet. How is this effect implemented?

I know little about the actual hardware and the networking that may or may not be involved.

Thanks!

Rituals before bed for better work life balance & sleep

Edwin Mak — Thu, 25 Apr 2019 20:41:59 +0000

Hey Community!

I've been having issues with keeping my mind free from thoughts about work while trying to goto and staying asleep. I've been meaning to practice mediation more on a daily basis as a supplement.

If you've had or have now a similar issue, what ways are you trying to get better rest?

Thanks!

The Useful & Hidden '_' Command In IRB

Edwin Mak — Thu, 07 Mar 2019 13:07:04 +0000

Introduction (TLDR sorta)

As a developer, I am alway seeking for tools and tricks that allow me to be more efficient while working on projects. As a result, I've adopted some lesser known ruby interactive shell command. One of which is the underscore or _. I found that this command when applied can be super beneficial and boost your productivity!

The underscore simply gives you the return value of the last evaluation made in the console. I find this especially useful in the event I forget to assign the output of a block of code or when I perform a query via Rails that I’d rather not run again. Let me show you what I mean:

Example Use Case

Above is a screenshot detailing an example use case. In that example, I evaluated a block of code with a 5-second pause (to simulate a slow operation) without assigning it to the variable animal. Nothing to worry about, you can just use the underscore to avoid having to execute that code again!

Another fairly common situation were you might benefit from using the _ method is when you run a query in Rails... but forgot to assign it to a variable!

Conclusion

The underscore is a useful tool to use when you are working in the ruby interactive shell. It may not seem like a lot of time to just re-run slow code… but it does add up!

What other kinds of hidden ruby tricks are there out? Please comment on this post if you know any and are in a charitable mood.