DEV Community: Arpan Kc

DynamoDB: the basics

Arpan Kc — Tue, 02 Jun 2020 04:09:36 +0000

Recently I've started looking at Serverless architecture inorder to move away from the monolithic architecture. While exploring AWS free tier, Amazon DynamoDB stood out to me. As a NOSQL database with free 25 GB of storage, I could use it for my personal projects basically for free.

So I've been reading up on AWS docs on the basics of DynamoDB, below listed are some of the basic terminology I have gotten to know about DynamoDB:

Tables

Strange name, given DynamoDB is NoSQL, However, it serves as a good mental model, and helps understand how data is grouped together. These 'tables' consists of a collection of 'items'.

Items

Items can be thought of as 'rows' in the tables, But unlike traditional tables, it does not have or need a specific schema. Aside from the primary key, it can contain any number of 'key-value' relations, and can also store nested objects upto 32 levels deep.

Primary Key

This is a simple one, Primary key uniquely identifies the 'item' in the table, it can be a single attribute or a combination of attributes.

Putting it all together

The image above is from AWS docs

In this specific case, 'People' is the table, which consists of a collection of items, And 'PersonID' is the primary key which uniquely identifies the item. Here, besides 'PersonID', other key value attributes can be anything, even nested items.

Conclusion

So AWS DynamoDB is an awesome free data storage option, however NoSQL Databases has its own use cases as does Relational Databases. So it is important to figure out what kind of storage option is best for your specific project before going forward.

Thanks for reading.

P.S. Please follow me on twitter @Nipeshkc7

How I've been misusing Async Await

Arpan Kc — Mon, 18 May 2020 03:27:27 +0000

Async-await is very intuitive and apparently very easy to misuse. Recently I've found out how I've been misusing this feature because of which my JS executes a lot slower than it should.

Let's look at the example to demonstrate what the problem was:

How I used to do it:

// Fetching promises one after the other, blocking execution

const getItems = async() =>{ 
    const itemA = await fetch("http://url.com/itemA");
    const itemB = await fetch("http://url.com/itemB");

    return [itemA, itemB]
}

How the pros do it:

// Fetching multiple promises concurrently to avoid delaying program

const getItems = async() =>{ 
    const itemA = fetch("http://url.com/itemA");
    const itemB = fetch("http://url.com/itemB");

    const items = await Promise.all([itemA,itemB]);
    return items;
}

See the core difference is how I was previously waiting for itemA to resolve before fetching itemB. On the other hand, when I wait for both the promises to resolve concurrently I speed up the execution by not having itemA to resolve before itemB.

(Note: However this is only recommended when the itemB does not somehow depend upon the promise from itemA being resolved first.)

P.S. Please follow me on twitter, I'd really appreciate it. @Nipeshkc

Deploying VueJS Front-end to Netlify

Arpan Kc — Mon, 27 Apr 2020 06:24:45 +0000

To preface, I'm currently working on a project whose backend API service I've recently published to heroku. So next step is to deploy my front-end.

Why Netlify?

Considering all the free options, I settled on Netlify. In addition to the Continuous deployments which let me deploy on every pull request, I also did not have to deal with any server side stuff. And unlike the Heroku free servers, Netlify (static)sites never sleep.

Deploying in 3 simple steps

Here's how I did it in 3 simple steps:

Added a netlify.toml in the root directory which consists of:

   [build]
     command = "npm run build"
     publish="dist"
     base = "client"

Because the front-end vuejs code is inside the 'client' subdirectory, I have to specify in this .toml file that the subdirectory to use is 'client' and the directory to publish after building is ./dist.

Added a _redirects file inside the /public directory inside client sub-directory.

   /*    /index.html   200

Include this file so that directly going to a different url (like
https://yoururl.netlify.app/home) won't return a '404')

And voila, netlify should build and deploy the site.

Thanks for reading.

Project Repo: https://github.com/nipeshkc7/BeatingBookies
Live project: https://beatingbookies.netlify.app/

P.S. Please follow me on twitter, I'd really appreciate it. @Nipeshkc

Deploying Node.js App to Heroku using Github Actions

Arpan Kc — Mon, 06 Apr 2020 00:04:10 +0000

Recently I started a project which was basically a Bet-tracking application with Node.js backend and Vue.js Frontend and Sqlite as the database. Initially I had planned on finishing the core functionality early and iterating often. So to do that, I decided to setup a CI/CD pipeline so that I could automate the testing and deployment. In this article I will talk about how I managed to setup a pipeline to automatically test and deploy my Node.js backend to Heroku using Github actions.

Initial Folder structure

So this is how my folder structure looks like (Hid other files for simplicity):

- Client    //Vue App
- Server    //NodeJs App

The Server consists of a Node.js server and Client consists of a VueJs app that sends API requests to the Server for CRUD functionality.

So in development environment this is how I would run my server:

cd server;
node index.js;

Setting Up Deployment pipeline

Creating a new deploy.yml file inside .github/workflows inside the root folder so the folder structure now looks like this:

- .github/workflows/deploy.yml
- Client    //Vue App
- Server    //NodeJs App

Get Heroku API key using the Heroku command line tool. Running the following command:

heroku authorizations:create

Add the following Github secrets in your github repo:
- HEROKU_API_KEY
- HEROKU_APP_NAME
Inside the deploy.yml I add the following:

name: Deploy

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

jobs:

  deploy:
      name: Deploy
      runs-on: ubuntu-latest

      steps: 
        - name: Checkout
          uses: actions/checkout@v1

        - name: Add remote origin
          run: git remote add heroku https://heroku:${{ secrets.HEROKU_API_KEY }}@git.heroku.com/${{ secrets.HEROKU_APP_NAME }}.git

        - name: Deploy backend to heroku
          run: git push --force heroku `git subtree split --prefix server HEAD`:master

To break everything down, here is how this yml file works:

Runs every pull request OR push to the repo.
Uses a pre-existing action called 'checkout'
Runs a job we named 'deploy' which adds a new remote heroku git repository
This command pushes the 'server' folder to heroku which contains our Node.js application:

git push --force heroku `git subtree split --prefix server HEAD`:master

The pipeline is setup, so basically now we have pushed the repo's server folder into heroku, Now we have to instruct heroku on how to handle the application. Before that add a ecosystem.config.js file inside the server folder which contains the following:

module.exports = {
    apps : [{
      name: "matched-betting-tracker",
      script: "./index.js",
      instances: "max",
      env: {
        NODE_ENV: "development",
      },
      env_production: {
        NODE_ENV: "production",
      }
    }]
  }

Define the start script inside the package.json of our server Node.js application.

"scripts":{
   // Other scripts
   "heroku-prebuild": "npm install pm2 -g",
   "start": "pm2-runtime start ecosystem.config.js --env production"
   }

This tells Heroku to install pm2 (which runs our nodejs app in production) before building the app and uses the 'start' script to run the application.

So next time you push to your repo, it should be automatically deployed to Heroku.

Feedbacks appreciated !

If you are interested in my current project, here's a link to my repo:

Matched-betting-tracker

Here's a link to an article where I write about implementing OAuth2.0 on my app.

OAuth 2.0 with Google API in Node.js

P.S. Please follow me on twitter, I'd really appreciate it. @Nipeshkc

Organizing my NodeJS code and folder structure

Arpan Kc — Sun, 02 Feb 2020 13:18:25 +0000

As I look at other people's nodeJS repository, I see neatly organized code and folder structure by people who seem like they really know what they're doing. Meanwhile I look at mine, it's a mess. As convenient as it might be to not having keeping everything in a single file, it's not the best practice. So I set on mission to perfectly organize my folder structure.

To do that, I went through a bunch of other people's repositories, read a bunch of articles and basically what I learned was there is no one-size-fits-all approach. However in my case, I did end up with a folder structure that I think will make my code somewhat maintainable.

In case of my project, I'm doing a full stack javascript application with vuejs at the front-end and nodejs at the backend.

App folder:
->client (consists of the client side VueJS)
->server
    --->controllers
    --->routes
    --->models 
    ---services
    --->helpers
    --->server.js (app entry point)
->.env / . git (and other stuff)

Here I'm only focusing on the server side nodejs. The main components are :

services: consists of methods that does the CRUD(Create Read Update Delete) dirty work
controllers: takes user request (HTTP requests in our case) and tells the services what to do
models: represents the database schema
routes : were done using express to connect urls (user's get and post requests) to controllers
helpers: Although there is no clear cut definition in terms of javascript, according to people programming in Java helpers are basically a bunch of functions that help in the internal working of the app, but do not directly serve the user's requests. So it would be safe to assume that it would do the same in Javascript.

So this is how you can organize you Nodejs folder structure for your project.

Thanks for reading and would would love to hear any insights into this matter.

Follow me on Twitter: @Nipeshkc*

Visit my blog: blog.arpankc.com

OAuth 2.0 with Google API in Node.js

Arpan Kc — Fri, 31 Jan 2020 11:45:03 +0000

With all the asyncs and the chainings and all the other JS shamaladingdongs , doing even simple tasks in Javascript can feel overwhelming, at least for a beginner like me.

To preface, I'm currently working on my own application which is a simple Match betting tracker to keep track of my bets, and decided to add in OAuth authentication in the app. Because why not.

The first thing I did was set up an account in Google console, and got me some shiny new credentials (client ID and client Secret) for OAuth 2.0.

I had trouble getting my head around the authentication flow with all the tokens and secrets and all that. But digging deep, it was quite simple actually and started to appreciate how google makes the process so simple and dare I say, elegant.

The first thing I did was get the google api:

  npm install googleapis

In the actual program:
initialized the api library and setup googleConfigs.


const { google } = require('googleapis');
const oauth2 = google.oauth2('v2');

const Oauth2Client = new google.auth.OAuth2(
    process.env.GOOGLE_CLIENT_ID,
    process.env.GOOGLE_CLIENT_SECRET,
    process.env.GOOGLE_REDIRECT, // this must match your google api settings
);

So basically all I had to do was these 3 things:

get a Redirect url for the clients
The url also contains a code which then I use to get a token ( if the user signs in to their google account that is)
And finally I use the token to get the user details

Step 1:

function getConnectionUrl() {
    return Oauth2Client.generateAuthUrl({
        access_type: 'offline',
        prompt: 'consent',
        scope: defaultScope
    });
}

//Call this function somewhere in the program to get URL:
//const url = getConnectionUrl();

Step 2 and 3:

function getUserDetails(code) {
    return Oauth2Client.getToken(code)   //use code to get the token
        .then(({ tokens }) => {
            Oauth2Client.setCredentials(tokens);     //add token to the Oauth credentials
        })
        .then(() => {
            return oauth2.userinfo.get({ auth: Oauth2Client });  // get userinfo with the newly updated credentials
        })
}

And so in the main program , I got a connection url using getConnectionUrl() which then gave me the code in the url which I passed on to getUserDetails() to get the user details.

So this is how I got OAuth authentication working in my application.

Thanks for reading. Would appreciate feedbacks.

If you want to check out my repo, here's the link :

Matched-betting-tracker

Please star it, if you like it.

P.S. Please follow me on twitter, I'd really appreciate it. @Nipeshkc

*UPDATE*

Using async-await steps 2 and 3 can be reduced to ::

 function async getUserDetails(code) {
   const {tokens} = await Oauth2Client.getToken(code);
   Oauth2Client.setCredentials(tokens);
   const usr_info = await oauth2.userinfo.get({auth: Oauth2Client});
   return usr_info;
 }

This looks cleaner and async await seems to be more intuitive to use.

What makes this site so quick?

Arpan Kc — Thu, 19 Oct 2017 17:49:13 +0000

I have been a frequent visitor of this site for more than a week now. And have to say I am amazed at how snappy this is. What is the secret? Could someone explain.

(I am sorry if this is a stupid question.)