DEV Community: mreigen

Elixir: Handy debug chain-able pretty-print function

mreigen — Mon, 08 Feb 2021 08:01:51 +0000

I picked up Elixir last year at work. While some debugging tools are good to casual debug things in Elixir, IO.puts/2 and IO.inspect/2 are sometimes much faster to show what I wanted to see in the console log output.

The drawback of IO.puts/2 is that it can only take a strings. It will fail if you pass in anything else, try a map or list! Using IO.inspect/2 is nice and all but I found that a lot of times the output just got lost in the logs.

I came up with a simple way to format the output so that it could be found easily in the logs by giving it a different color than the typical green or blue output text and set a label to each output so that in the cases I have more than one output at a time, I would know which one is for which.

In the example below, you can see how simple it is to use. I’m using it with a map in this case. I named the function D.pp\2 (in the D module in my project by the way.)

As you can see it returns the map to be chained:

%Membership{}
|> changeset(params)
|> put_assoc(:stakeholders, [])
|> D.pp("Changeset after put assocs stakeholders") #<-------
|> touch(user)
|> Repo.insert()

In the example above, D.pp/2 is put in between the pipes to show the value of the changeset after put_assoc so I could see what is the changeset value at that exact point, and wouldn’t mess up the data in the workflow.

Or I could use it in tests:

response = build_conn()
|> as_user(context.regular)
|> get(context.path)
|> D.pp("after get") #<--------
|> json_response(200)
|> D.pp("received response") #<---------

Here is the function!

def pp(what_to_print, debug_message \\ "") do
  IO.puts("#{IO.ANSI.yellow()} START #{debug_message} ====================")
  IO.puts(IO.ANSI.yellow() <> inspect(what_to_print, pretty: true, limit: :infinity))
  IO.puts("#{IO.ANSI.yellow()} END   #{debug_message} ====================\n\n")
  what_to_print
end

Maybe the next step is to create its own module and allow users to configure settings for each project for color, infinity limit, etc…? You can simply put this in your project in a module named D or Debug and call it like how I call it D.pp

Please let me know in the comment if you have any question and if you find it useful in your project, please help me out by letting me know here, mention me somewhere and/or hit that love button! Thank you.

Build a Chat system using Rails 5 API Action Cable and ReactJS with multiple private rooms and group chat option

mreigen — Sat, 06 Feb 2021 19:33:07 +0000

Please note that this post is not a tutorial and it requires knowledge of Rails 5 ActionCable and ReactJS / Javascript custom library building.

(please note that this short post will not show you how to build this front-end component though)

One of the awesome features that comes with Rails 5 is ActionCable. With ActionCable, you can build all the real-time features you can think of via websocket. While struggling to build a chat system, I had found multiple examples on the ‘net of how to build a chat app with Rails 5 ActionCable but they are extreme simple to even apply the concept for any real life chat application. I believe this is the first example on the internet that shows you how to build such a chat system with:

Rails 5 API backend and a ReactJS frontend
Multiple private rooms
Any positive number of users in a room (not just 1–1) or group chat

The chat system my talented friend Tim Chang and I have built has:

Multiple private chat rooms
Multiple chat users per room
Online / Offline status of each user
Real-time “typing…” status
Real-time read receipt

In this short post, I’ll show you only the basic of #1 and #2. Please leave me a comment below if you want me to show you how to build #3, #4 and #5. I’m using Rails 5 as the back-end API and ReactJS library on the front-end.

Backend

On creation, Rails will generate the channels folders and files where all the real-time magic happens :)

app/channels/application_cable/channel.rb
app/channels/application_cable/connection.rb

Authentication

First of, let’s authenticate the websocket connection requests to your Rails server inside connection.rb.

module ApplicationCable
  class Connection < ActionCable::Connection::Base
    identified_by :current_user

    def connect
      self.current_user = find_verified_user
    end

    private
      def find_verified_user
        # or however you want to verify the user on your system
        access_token = request.params[:'access-token']
        client_id = request.params[:client]
        verified_user = User.find_by(email: client_id)
        if verified_user && verified_user.valid_token?(access_token, client_id)
          verified_user
        else
          reject_unauthorized_connection
        end
      end
  end
end

Depending on the authentication gem or service that you use in your project, find_verified_user method should be modified to your need. I have a method called valid_token? to verified the access-token and client_id passed in with the websocket request. If the request is not authenticated, then it will be rejected.

Data Structure

The idea is very basic: a chat room that has multiple messages, each message has a content and a sender. Note that a message doesn’t have a “receiver”. This allows a room to have any number of users since you don’t need to care about the receiver of the messages, since all the messages from the senders will end up appearing in a room regardless of how many participants in the room. So, this is the data structure that I use:

Conversation (room): has_many messages, users and has an id
Message: belongs_to a conversation, has a sender, has the text content
Sender: is a User

As a result, I created 3 models:

# message.rb
class Message < ApplicationRecord
  belongs_to :conversation
  belongs_to :sender, class_name: :User, foreign_key: 'sender_id'

  validates_presence_of :content

  after_create_commit { MessageBroadcastJob.perform_later(self) }
end

# conversation.rb
class Conversation < ApplicationRecord
  has_many :messages, dependent: :destroy
  has_and_belongs_to_many :users
end

# user.rb
class User < ApplicationRecord
  has_and_belongs_to_many :conversations, dependent: :destroy
end

Action triggers

When a client connects (subscribed) or broadcasts a message (speak), the backend will react with actions. Inside folder app/channels, I will create a file called room_channel.rb.

# room_channel.rb
class RoomChannel < ApplicationCable::Channel
  # calls when a client connects to the server
  def subscribed
    if params[:room_id].present?
      # creates a private chat room with a unique name
      stream_from("ChatRoom-#{(params[:room_id])}")
    end
  end

  # calls when a client broadcasts data
  def speak(data)
    sender    = get_sender(data)
    room_id   = data['room_id']
    message   = data['message']

    raise 'No room_id!' if room_id.blank?
    convo = get_convo(room_id) # A conversation is a room
    raise 'No conversation found!' if convo.blank?
    raise 'No message!' if message.blank?

    # adds the message sender to the conversation if not already included
    convo.users << sender unless convo.users.include?(sender)
    # saves the message and its data to the DB
    # Note: this does not broadcast to the clients yet!
    Message.create!(
      conversation: convo,
      sender: sender,
      content: message
    )
  end

  # Helpers

  def get_convo(room_code)
    Conversation.find_by(room_code: room_code)
  end

  def get_sender
    User.find_by(guid: id)
  end
end

As you can see in the comment, after a client “speaks”, the broadcasting is not happening yet; only a new Message is created with its content and data. The chain of action happens after the Message is saved in the DB. Let’s take a look again in the Message model:

after_create_commit { MessageBroadcastJob.perform_later(self) }

Scalability

This callback is called only after the Message is created and committed to the DB. I’m using background jobs to process this action in order to scale. Imagine that you have thousands of clients sending messages at the same time (this is a chat system, why not?), using background job is a requirement here.

# message_broadcast_job.rb
class MessageBroadcastJob < ApplicationJob
  queue_as :default

  def perform(message)
    payload = {
      room_id: message.conversation.id,
      content: message.content,
      sender: message.sender,
      participants: message.conversation.users.collect(&:id)
    }
    ActionCable.server.broadcast(build_room_id(message.conversation.id), payload)
  end

  def build_room_id(id)
    "ChatRoom-#{id}"
  end
end

Here is when the broadcasting happens. ActionCable will broadcast the payload to the specified room with the provided payload.

ActionCable.server.broadcast(room_name, payload)

Cable Route

You will need to add the /cable websocket route to your routes.rb so that your client can call this endpoint to broadcast and receive messages.

mount ActionCable.server => '/cable'

And that’s it for the backend side! Let’s take a look at the ReactJS front-end library.

Client Library

Please note that depending on the specifics of your project, you will need to understand the concept of this code in this library and modify it to your needs.

First, install the ActionCableJS via npm.

Create a ChatConnection.js file as one of the services in your ReactJs app.

// ChatConnection.js

import ActionCable from 'actioncable'

import {
  V2_API_BASE_URL,
  ACCESS_TOKEN_NAME,
  CLIENT_NAME,
  UID_NAME
} from '../../globals.js'

function ChatConnection(senderId, callback) {
  let access_token = localStorage.getItem(ACCESS_TOKEN_NAME)
  let client = localStorage.getItem(CLIENT_NAME)

  var wsUrl = 'ws://' + V2_API_BASE_URL + '/cable'
  wsUrl += '?access-token=' + access_token + '&client=' + client

  this.senderId = senderId
  this.callback = callback

  this.connection = ActionCable.createConsumer(wsUrl)
  this.roomConnections = []
}

ChatConnection.prototype.talk = function(message, roomId) {
  let roomConnObj = this.roomConnections.find(conn => conn.roomId == roomId)
  if (roomConnObj) {
    roomConnObj.conn.speak(message)
  } else {
    console.log('Error: Cannot find room connection')
  }
}

ChatConnection.prototype.openNewRoom = function(roomId) {
  if (roomId !== undefined) {
    this.roomConnections.push({roomId: roomId, conn: this.createRoomConnection(roomId)})
  }
}

ChatConnection.prototype.disconnect = function() {
  this.roomConnections.forEach(c => c.conn.consumer.connection.close())
}

ChatConnection.prototype.createRoomConnection = function(room_code) {
  var scope = this
  return this.connection.subscriptions.create({channel: 'RoomChannel', room_id: room_code, sender: scope.senderId}, {
    connected: function() {
      console.log('connected to RoomChannel. Room code: ' + room_code + '.')
    },
    disconnected: function() {},
    received: function(data) {
      if (data.participants.indexOf(scope.senderId) != -1) {
        return scope.callback(data)
      }
    },
    speak: function(message) {
      return this.perform('speak', {
        room_id: room_code,
        message: message,
        sender:  scope.senderId
      })
    }
  })
}

export default ChatConnection

So here is the hook: in createRoomConnection, the client will try to connect with (subscribe to) the RoomChannel we created in the backend, once it’s connected (subscribed), it will stream from the room name ChatRoom-id (look at room_channel.rb above again.) Once it’s connected, there are 2 methods that will be called frequently, can you guess which one?

They are: received and speak!

The received method is called when there is a message broadcast to the client from the server, on the opposite, speak is called when the client broadcasts a message to the server.

Voila! That’s it. Again, this is not made to be a ready-to-run-out-of-the-box kind of tutorial because each project is different, but I hope it gives you an idea how to build a chat system with multiple private chat rooms and multiple users per room. Please let me know in the comment section if you have any question.

And please don't forget to hit the love button if you find this helpful to your project!

Elixir: A cleaner way to organize tests using ExUnit's named setup.

mreigen — Sat, 06 Feb 2021 08:10:13 +0000

The dev team at Vendorful take code quality seriously. We invest in writing tests for every bug fix and new feature being implemented and in well reviewed merge requests. As any good developer should know from their past experience that with good test coverage they can actually move faster as a team in the long run. However, in addition to having good test coverage, writing human-readable code and tests is equally important. Personally, my feeling of having to work with a neatly nested and organized test suite set up but extremely difficult to follow is almost comparable to having to walk through a wild maze to find something that is not very worthwhile, every time.

Coming from Ruby on Rails background, I used to use RSpec to write my tests. While coding with Elixir and using ExUnit to write tests nowadays, I caught myself finding ways to nest RSpec context blocks inside each other. Let's find a simple example: you wanted to build a chef robot to prepare you dinners base on the left over food in your fridge. Depending on what's available in your kitchen, it'll choose how to prepare the appropriate dishes.

Nested contexts in RSpec

This is how I would write TDD tests for this scenario in RSpec.

describe "make_dish_from_left_over" do
  context "with dishes that are served cold" do
    # before: set up cold dishes
    # ...
    # end
    context "seasoning available" do
      # before: set up seasoning
      # ...
      # end
      expect(add_seasoning).to be_truthy
    end
    context "extra protein available" do
      # before: set up extra protein
      # ...
      # end
      expect(add_protein).to be_truthy
    end
  end

  context "with dishes that are served hot" do
      # before: set up hot dishes
      # ...
      # end
    context "microwave available" do
      # before: set up microwave
      # ...
      # end
      expect(warm_up_by_microwave).to be_truthy
    end
    context "microwave NOT available" do
      # before: set up stove top
      # ...
      # end
      expect(warm_up_by_stove_top).to be_truthy
    end
  end
end

As you can imagine, if these tests are all filled up and more nested contexts are added, such as: microwave settings depending on size of dishes, stove top heat amount and time, etc... it will become too long and too complicated to read and follow, especially for other devs who haven't worked on this area of the robot. Also, there is no way to write nested "contexts" like this in Elixir's ExUnit - and there is a good reason for that.

By forbidding hierarchies in favor of named setups, it is straightforward for the developer to glance at each describe block and know exactly the setup steps involved. - ExUnit documentation

With ExUnit, what you can, and want to do is to use named setups.

What are named setups?

I have to admit that the documentation for named setup in ExUnit is kind of buried somewhere in the middle of ExUnit.Case documentations. Named setups are simply configurations of tests used to put together test data for the following tests. Each setup is actually just a function that configures the test data and returns the variables to be used in the tests themselves. If you are familiar with writing tests in ExUnit then these configuration functions are exactly the setup blocks. For example:

defp setup_cold_dishes(context) do
  temp = 35
  size = if context.size, do: size, else: :small
  dish = set_temperature(temp)
  |> set_size(size)
  |> set_time_in_fridge(:15_hours)
  |> build_dish

  %{temp: temp, size: size, cold_dish: dish}
end

Note that a context variable is passed in this setup function so that the previous configurations can be accessed in this setup as context.size in this case. You can also grab the size variable out of this context right in the function signature like so:

defp setup_cold_dish(%{size: size}) do
end

How do I use it in the tests?

describe "make COLD dishes from left over" do
  setup [:setup_cold_dish]
  it "", context do
    ...
  end
end

Let's make it even nicer by combining the condition "seasoning available":

describe "make COLD dishes from left over when seasoning available" do
  setup [:setup_cold_dish, :set_seasoning]
  it "", context do
    assert add_seasoning
  end
end

Passing variables to other setup functions

Note that if you have multiple setup functions in sequence like this, the later ones on the right can read the "context" variables returned by the previous ones on the left. So :set_seasoning can actually access temp, size and cold_dish variables from :setup_cold_dish but we just not doing anything with them in this scenario.

Putting the pieces together

Now let's write the whole thing with combined conditions!

describe "make COLD dishes when seasoning available" do
  setup [:setup_cold_dish, :set_seasoning]
  it "", context do
    assert add_seasoning
  end
end

describe "make COLD dishes when extra protein available" do
  setup [:setup_cold_dish, :set_protein]
  it "", context do
    assert add_protein
  end
end

describe "make HOT dishes when microwave available" do
  setup [:setup_hot_dish, :set_microwave]
  it "", context do
    assert use_microwave
  end
end

describe "make HOT dishes when stove top available" do
  setup [:setup_hot_dish, :set_stove_top]
  it "", context do
    assert use_stove_top
  end
end

Conclusion

The tests are now flattened, much cleaner and easier for other developers (and for me at a later time) to read and follow at a glance. And the beauty of it is: no more nested context/describe blocks, no more mazes but with shared setup functions and cleaner code!

Although the above example scenario is kept very simple to demonstrate the basic functionalities of named setups, but I hope you could get the idea and apply it to your test suite when you need it.

How to restore (recreated) a deleted remote git branch

mreigen — Thu, 04 Feb 2021 06:29:13 +0000

I ran into this when I was cleaning up my local git. I deleted both a local and its remote branch on origin! 🤦‍♂ And the branch I deleted is still in a Merge (Pull) Request! 💀 I used a third party app called SourceTree (I’m sure you heard of it) to delete them, so my commit is not in the reflog

What I did to restore the local branch is as follow:

Since my commit is not in the reflog, I had to do this to print out and ultimately find my commit’s sha

git fsck --full --no-reflogs --unreachable --lost-found | grep commit | cut -d\ -f3 | xargs -n 1 git log -n 1 --pretty=oneline > .git/lost-found.txt

I could open this file with my editor to find the sha by my commit message (I was glad I remembered some key words in it) and made sure if the changes are what I did:

git log -p <sha>

Finally, I recreated the deleted branch by doing:

git branch hotfix-whatever-branch <sha>

All I did to do after this is to push the branch and voila, the branch was restored successfully!

What I learned from this syntax:

git fsck --full --no-reflogs --unreachable --lost-found | grep commit | cut -d\ -f3 | xargs -n 1 git log -n 1 --pretty=oneline > .git/lost-found.txt

--git fsck: file system check. It verifies the connectivity and validity of the objects in the database

-no-reflogs Do not consider commits that are referenced only by an entry in a reflog to be reachable. This option is meant only to search for commits that used to be in a ref, but now aren’t, but are still in that corresponding reflog

--unreachable: Print out objects that exist but that aren’t reachable from any of the reference nodes.

--full Check not just objects in GIT_OBJECT_DIRECTORY ($GIT_DIR/objects), but also the ones found in alternate object pools listed in GIT_ALTERNATE_OBJECT_DIRECTORIES or $GIT_DIR/objects/info/alternates, and in packed Git archives found in $GIT_DIR/objects/pack and corresponding pack subdirectories in alternate object pools. This is now default; you can turn it off with --no-full.

--lost-found Write dangling objects into .git/lost-found/commit/ or .git/lost-found/other/, depending on type. If the object is a blob, the contents are written into the file, rather than its object name.

You can see the full documentations here:
https://git-scm.com/docs/git-fsck

The rest of the pipeline is to pick the info we need, format it and send it to a file.

The Linux cut command https://www.geeksforgeeks.org/cut-command-linux-examples/

ES6 dynamic key (name) for object & object property assignment

mreigen — Thu, 04 Feb 2021 06:21:31 +0000

Hands up! Or I will dynamically access your property.

Recently, I’ve picked up Javascript again at my day job writing software using ReactJS and Elixir. It had been a while I hadn’t code javascript professionally since AngularJS 1. Since 2014, there are so many new changes to the JS libraries and communities! I need to catch up.

Today I learned the new way in ES6 to use a dynamic key to access or assign an object property.

ES6 enables developers to create or access an object by dynamic keys or names:



const key1 = "make";
const key2 = "model;
const newObj = {
  year: 2020,
  [key1]: "toyota"
  [key2]: "prius"
}

You can think of many ways you can apply this to your coding scenario. How about the case where you might need to create an object with an increasing number in the key names?



const name = "com";
let i = 1;

const radioDevice = {
  numberOfComs: 3,
  [name + "_" + i++]: "port 4556",
  [name + "_" + i++]: "socket 12",
  [name + "_" + i++]: "socket 15",
};

radioDevice.com_1; 
// => port 4556

Or, when you want to replace an item in an array with a dynamic index 😉



let array = [
  {name: "Minh", age: 20},
  {name: "Brian", age: 22},
  {name: "Hugo", age: 12},
  {name: "Zelda", age: 56},
  {name: "Simon", age: 7}
];

const nameToFind = "Hugo";
const personToReplace = {name: "Ali", age: 34};

const index = array.findIndex(item => item.name === nameToFind);
Object.assign([], array, { [index]: personToReplace });
//=> [
// {name: "Minh", age: 20}
// {name: "Brian", age: 22}
// {name: "Ali", age: 34} <---------
// {name: "Zelda", age: 56}
// {name: "Simon", age: 7}
//];

ES6's Object.assign function is used to copy the values of all of the enumerable own properties from one or more source objects to a target object

ReactJS: auto save feature for any input field

mreigen — Thu, 04 Feb 2021 06:15:47 +0000

I picked up ReactJS recently after a few years away from coding in Javascript at all. Since ReactJS is a powerful and useful framework with a lot of bells and whisles, in the midst of so many new things to learn, I lost sight of it being just a javascript framework. Yes, I can still use vanilla JS in my ReactJS app! (duh)

Here is one example:

My team wanted to implement a simple debounced auto-save function for our text box input. At first we used lodash's debounce to make it work. It seemed to be working and we launched it. But customers came back with a complaint that typing in the textbox appeared to be jittery. So, here is the old code:



import { debounce } from "lodash";
import RichText from "../controls/rich_text";

...

class TextQuestion extends React.Component {
  constructor(props) {
    super(props);
    this.update = debounce(this._update.bind(this), 500);
  }

  _update(text) {
    this.setState({ text });

    callAPIToSaveText({ answer: text });
  }

  ...

  render() {
    ...

    <RichText.NoToolbar editorClass="small" value={this.state.text} onChange={this.update} />
  }
}

This works when it doesn’t :) e.g. when a user is typing with an interval of about half a second between letters. The app would save the text, and re-render the textbox with the saved text while discarding what they have typed after the saving.

To fix the bug, I opted to using vanilla JS’s setTimeout as follow:



import RichText from "../controls/rich_text";

const SavingState = Object.freeze({
  NOT_SAVED: 0,
  SAVING: 1,
  SAVED: 2
});

class TextQuestion extends React.Component {
  constructor(props) {
    super(props);

    this.state = {
      text: "",
      saving: SavingState.NOT_SAVED
    };

    this.update = this.update.bind(this);
  }

  componentDidMount() {
    this.timer = null;
  }

  update(text) {
    clearTimeout(this.timer);
    this.setState({ text, saving: SavingState.NOT_SAVED });

    this.timer = setTimeout(() => {
      this.setState({ text, saving: SavingState.SAVING });

      // calls API
      callAPIToSaveText({ answer: text })
      .then(() => this.setState({ saving: SavingState.SAVED }));
    }, 500);
  }

  render() {
    <RichText.NoToolbar editorClass="small" value={this.state.text} onChange={this.update} />
    <AutoSaveDisplay saving={this.state.saving} />
  }
}

const AutoSaveDisplay = ({ saving }) => {
  let display;
  switch (saving) {
    case SavingState.SAVING:
      display = <em>saving...</em>;
      break;
    case SavingState.SAVED:
      display = (
        <>
          <Icon check /> <em>saved!</em>
        </>
      );
      break;
    default:
      display = <br />;
  }
  return <div className="auto-save-display">{display}</div>;
};
AutoSaveDisplay.propTypes = {
  saving: PropTypes.oneOf(Object.values(SavingState))
};

// styles I use for auto-save-display
// .auto-save-display {
//  display: block;
//  height: 25px;
//}

Here we go. Auto-save works now! The AutoSaveDisplay component you saw above is only a cherry of top bonus to display “saving…” and “saved” when it’s saving your text.