DEV Community: Manoj Naidu

Rails Deployment to AWS EC2

Manoj Naidu — Thu, 08 Oct 2020 12:46:54 +0000

Hello developers! A link to the detailed video explaining the deployment process and additional resources are attached below. Please ❤️ this post if you like the content.

manojnaidu619 / manojnaidu619

Namaste 🙏

I'm Manoj Naidu, just a simple dude 😁 who loves to build things.

Tech & Tools ⚙️

Github Stats 📈

Reach me

View on GitHub

AlphaVantage Action Bot - renders realtime chart of a stock/cryptocurrency inside Readme

Manoj Naidu — Thu, 17 Sep 2020 11:41:20 +0000

Hello programmers, hope you are all doing fantastic!🥳 .We all know that Readme file is crucial for any repository, it serves as a user manual and helps us in getting started.

This action makes the best use of Readme file to show the trajectory of stock prices in order to gauge your company’s general health.

What I Built?

AlphaVantage Action Bot is a Github action that renders the realtime chart of a stock/cryptocurrency inside the readme file. We will be using the Alpha Vantage Public Stock APIs to build the action bot. You may also check out this finance API guide that surveys some of the popular stock and crypto data APIs in the market.

Submission Category:

Wacky Wildcards

Screenshots

BITCOIN open price trajectory

close price trajectory of APPLE

Market Capital(USD) of BTC

How to Setup?

This action could be adopted easily into your project in just 3 steps...

Step 1 - Adding script file into your repo.

This step is pretty simple, all you need to do is..

Create folder named alphavantage in the root of your repo.
Inside alphavantage folder, add a new python script file with name alphavantage-bot.py and paste the complete code below

From line 8-50, the variables could be customized to requirements.

So now, your repo will contain a python script file, whose path is YOUR_REPO_NAME/alphavantage/alphavantage-bot.py. It should look like this...

Step 2 - Adding AlphaVantage API key to repo secrets.

Since we are using AlphaVantage's API to get the data, we need to obtain API key from them to utilize their service. Click here to get API key.

After filling the basic details, you should get your API key instantly.

Now, paste the API key inside your repo secrets with the name ALPHA_VANTAGE_KEY.

Step 3 - Setup Workflow file(Final step, yay!).

Paste the below code inside
REPO_ROOT/.github/workflows/AlphaVantage-Action-Bot.yml


name: AlphaVantage-Action-Bot

on:
  schedule:
    - cron: "*/30 * * * *"  
    # Runs every 30th minute
    # Could be customized to requirement
    # To know more about crontabs refer https://crontab.guru/ 


jobs:
  alphavantage-render-chart:
    runs-on: ubuntu-latest
    steps:
      - name: checkout repo content
        uses: actions/checkout@v2 # checkout the repository content to github runner.
      - name: setup python
        uses: actions/setup-python@v2
        with:
           python-version: 3.8 #install the python needed
      - name: Install dependencies
        run: python -m pip install --upgrade pip matplotlib pandas alpha_vantage
      - name: execute python script 
        run: python ./alphavantage/alphavantage-bot.py
        env:
           ALPHA_VANTAGE_KEY: ${{ secrets.ALPHA_VANTAGE_KEY }}
      - name: Commit and push if changed
        run: |
         git add .
         git diff
         git config --global user.email "alphavantage-action-bot@example.com"
         git config --global user.name "AlphaVantage Action Bot"
         git commit -m "AlphaVantage Action Bot Updated README" -a || echo "No changes to commit"
         git push

Now, your workflow file's path should like something like this...

Congrats! You have successfully set up the workflow 😎

Link to Code

manojnaidu619 / AlphaVantage-Action-Bot

A github action to render real-time stocks/cryptocurrency charts inside readme

AlphaVantage-Action-Bot

A github action to render real-time stocks/cryptocurrency charts inside readme

**Realtime Stock/Crytpocurrency Chart📈 Rendered By AlphaVantage-Action-Bot | Last updated the above chart on Oct 07, 2024(00:57:34) **

Hello programmers, hope you are all doing fantastic!🥳 .We all know that Readme file is crucial for any repository, it serves as a user manual and helps us in getting started.

This action makes the best use of readme file to show the trajectory of stock prices in order to gauge your company’s general health.

📌 Click on DEV logo below to view detailed article

What Is it?

View on GitHub

Additional Resources / Info

Special Thanks to the AlphaVantage team for reaching me out on this article✌️

BookMyGIG - live gigs for live audiences

Manoj Naidu — Thu, 04 Jun 2020 19:34:13 +0000

As the pandemic broke out, Shows/events were called off across the globe. This has impacted the creators severely and made them sit idle at home. This is when I thought of building something which helps creators make some penny online from their fresh content.

BookMyGIG is a platform where creators/artists perform live online shows(could be related to Dance, Fitness, Comedy, Plays, Wokshops, Concerts and the list goes on...) for live audiences. There is also a chat section, where the users who are part of the same show can chat in realtime while they are watching it.

Submission Category:

Exciting Experiments

Demo

Hompage with monetization support

Creator gig registration

Streaming credentials

Streaming page with no monetization

Streaming page with monetization

Stream session

Realtime chat demo

How it Works in a nutshell

Creator's perspective
- Registers for a new show/event by filling out gig registration form, where finally needs to provide wallet pointer to accept the micro-payments.
- After successful registration, OBS credentials are provided to start the live stream.
User's perspective
- Goes through the list of registered shows/events on homepage.
- Gets tickets/access to gigs by making successful online payment.
- Sits back and enjoys the live show 🍿🥤.
- Not just that, users who are part of the same show can chat with each other in realtime with the help of realtime chatbox.
But, how's monetization working under the hood?
- When the user is in streaming page (creator's live stream section), the micro-payments are sent to creator's wallet 💰.
- When the user is anywhere in the site other than creator's streaming page, BookMyGIG is making money 💵.

index.html looks like...

  <!DOCTYPE html>
  <html>
   <head>
     .....
     <meta name="monetization" content="$BOOKMYGIG_WALLET_POINTER">
     .....
   </head>
   <body>
     .....
   </body>
  </html>

and the JS code looks something like this...

window.addEventListener('load', () => {
  const inStreamPage = window.location.pathname.startsWith('/gig/')
  if(inStreamPage){
    const metaTag = document.querySelector('meta[name="monetization"]')
    metaTag.setAttribute("content", '$CREATOR_WALLET_POINTER')
  }
})

How I built it

Framework/Technologies used :

ReactJS
NodeJS
Redis as an in-memory database
Node-Media-Server for video streaming
Socket.io for realtime-chat
Web Monetization API + Coil for monetizing content

4 Pillars of the application :

REDIS is used to store data, as it is an in-memory database which makes our app incredibly fast and the process of exchanging data back and forth is seamless. Our application uses different blend of built-in data structures to store and retreive data in an efficient manner.
RTMP provides a bidirectional message multiplex service over a reliable stream transport, such as TCP, intended to carry parallel streams of video, audio, and data messages, with associated timing information, between a pair of communicating peers. More about RTMP could be learned here.
- When a creator lists a gig, he/she is given a unique streaming ID which is used to identify the creator on the backend and allocate a separate channel, where creator could live stream and also the audiences of that particular show are isolated from the rest of the channels/shows.
- As soon as the creator hits start stream button, the video data is transported to media server, where it is encoded to different formats. In our case we use flv, which is a file format used by Adobe Flash Player to store and deliver synchronized audio and video streams over the Internet.
- Later, on the client side we use a flvjs plugin to render the video in realtime.
REALTIME CHAT is accomplished using socket.io, which is a library to abstract the WebSocket connections. It enables realtime, bi-directional communication between web clients and servers.
- When a client types the message and clicks send, it is sent to server and is then broadcasted to all the connected clients in the same room.
- The messages that gets exchanged within a room is isolated from the outside world.

Also, we are using Redis pubsub indirectly as socket.io internally relies on it to achieve the realtime two-way communication.

Web-Monetization gives a special flavour to our product. Apparently, this is one of the best ways to monetize the content by not annoying the users with those flashy banner ads. Let us see how it is being used?
- Creator makes money when...
  - User buys the access to the show.
  - User Streams the show(through micro-payments).
- BookMyGIG makes money when...
  - User is anywhere in the site(Other than the show streaming page).

Challenges I ran into

Handling creator data in the backend and storing it efficiently using built-in Redis data structures.
Fetching all the shows asynchronously(using promises), and resolving them was tricky.
Figuring out a way to load & play the live stream on the browser without clashing between other sessions was really challenging.

Accomplishments that I'm proud of

I could successfully able to hook-up all the different parts of the application together and come up with the working end product.

What I learned

This was the first time having my hands-on experience with Redis, Socket.io, RTMP, Web-Monetization...It was a lot of fun 😁

Link to Code

manojnaidu619 / bookmygig-frontend

ReactJS frontend of bookmygig

BOOKMYGIG

This is the frontend part, checkout the backend here

📌 Click on DEV logo below to view detailed post

Inspiration

How it Works in a nutshell

Creator's perspective
- Registers a new show/event by filling out gig registration form, where finally needs to provide wallet pointer to accept the micro-payments.
- After successful registration, OBS…

View on GitHub

Additional Resources/Info

Whatsupp SSH? - accessing SSH over WhatsApp

Manoj Naidu — Sun, 26 Apr 2020 18:43:34 +0000

We all wanted to have this superpower of controlling anything from anywhere, but that's not possible yet! (maybe someday in the future...) But today I am joyful to showcase my first ever dev.to hackathon project, which doesn't open the doors to rule every single thing, but certainly SSH / Remote Server over WhatsApp.

While I was doing my internship, I used to SSH into EC2 many times a week, and to do that I always needed to have my laptop handy. This is not a thing to be worried about. But... when you seriously need to access your remote server for some important update which cannot be postponed and you realize that your laptop is resting at your home. That "oh No!" moment hurts a lot...

So, then I started to look for alternatives for accessing my EC2 without leaning towards my laptop always. Then eventually, my internship came to an end, but that "search for alternatives" was still hanging in my mind.

Recently, I started learning NodeJS and this is when I also came across #twiliohackathon tag on dev. I had no idea about Twilio until I had a look at their wide range of APIs and cool web services offered by them.

After digging deep into Twilio's services. Finally, that "search for alternatives" got a slight spark.

So, What I built?

I built a Nodejs application integrated with Twilio's API for WhatsApp, which could be installed and configured on any remote server(dead simple to setup!, trust me 🙌) or computer(with UNIX based OS), results in gaining access to it remotely and execute shell commands over WhatsApp.

Category Submission

Exciting X-Factors

Demo

✅ Custom Authentication

✅ brew-update over Whatsapp

✅ Executing git commands

✅ mkdir over whatsapp

✅ executing Python script

✅ Demonstrating custom command ssh-help

✅ Demonstrating custom command ssh-reset (to reset the working directory)

✅ Demonstrating custom command ssh-status (to retreive system status and extra info)

✅ Demonstrating custom command ssh-history (alias version of history bash command. But here it lists the commands executed over Whatsapp)

How does it work?

PHASE-1 ➜ The command which we need to execute on the server is sent to Twilio.

PHASE-2 ➜ Twilio forwards the request to our app. For this particular action to work, we need to setup a webhook inside twilio console...(We will talk about this in Setup section of this post).

PHASE-3 ➜ After receiving the request from Twilio, Our app first verifies that the request is actually being sent by Twilio. Otherwise, the request would be rejected. Then, it executes the command entered by the user and sends back the output/response in the format which is understood by Twilio (Twilio Markup Language(TwiML))

PHASE-4 ➜ Once Twilio receives back the TwiML response from our app, it sends it back to the user.

What's the Stack?

Pure NodeJS

📍 But, to setup and get started we need...

Valid Twilio account
A remote server/ computer (on which we could execute shell commands, I am using AWS EC2)

How to Setup?

The setup process is really simple, you just have to follow these four steps...

✏️ STEP-1. Signup for a Twilio Account and join the Twilio Whatsapp sandbox.

Signup for an account here

Now, login and join the sandbox by doing as directed on screen and complete all 3 steps. Don't share your sandbox code with anyone (The red block covers my sandbox code)

One final thing needs to be added into Twilio. We will see that later...

✏️ STEP-2 Configure port on the server/ computer.

If you are setting up in the local computer, then you are free to skip to STEP-3.
If setting up in a remote server, then you need to configure the instance/ droplet to open port 3003 for incoming requests.

📌 port 3003 is where Twilio would be forwarding the requests to...

If using AWS EC2 then you need to add a new rule inside Security Groups -> Inbound rules of a particular instance.

Then add a new rule like so...

📌 If using other than EC2, then refer to official docs.

✏️ STEP-3 Let's move towards our computer/ server.

📌 All the actions from now are performed inside the terminal.

cd into the directory where you want to clone the app.
Now, clone the project repo.

manojnaidu619 / Whatsupp-SSH

Access your SSH over Whatsapp

$ sudo git clone https://github.com/manojnaidu619/Whatsupp-SSH.git

cd into the project folder and run sudo npm install

$ cd Whatsupp-SSH/ 

$ sudo npm install

As we are logging the requests into a log file, we need to give appropriate permissions to the app directory and the folders inside it. (The path to project must be absolute)

$ sudo chmod -R a+rw ~/home/Whatsupp-SSH

Now adding env variables, which our app relies on. Make sure the key is same as mentioned below.

📌 Here, I am considering Ubuntu as the OS.

$ sudo nano /etc/bash.bashrc

scroll down to the bottom of the file and add these lines by replacing the values.

export SSH_PSWD=YOUR_DESIRED_PASSWORD
export TWILIO_URL=http://PUBLIC_IP_OF_SERVER:3003/Whatsupp-SSH
export TWILIO_AUTH_TOKEN=YOUR_TWILIO_AUTH_TOKEN

then source the bash.bashrc file by typing.

$ source /etc/bash.bashrc

Now, copy the same TWILIO_URL that was added to bash.bashrc file.

📌 Remember that we had one last thing to add to Twilio sandbox configuration... It's time to do that.

head to twilio console -> programmable SMS -> Whatsapp -> Sandbox

After adding that, scroll down and hit Save.

✏️ STEP-4. Head to your server/computer and run these final commands.

install pm2.

$ sudo npm install pm2 --global

Now, run pm2 startup to initialize startup scripts. So, whenever the server reboots/ crashes, our node app would also be picked up automatically.

$ pm2 startup

Now copy-paste the command given by pm2 (the one outlined by red border) and hit enter.

Now, to save them all run pm2 save.

$ pm2 save

just one final command left, you have successfully setup the app. Now let's start the pm2 server.

$ pm2 start ABSOLUTE_PATH_TO_WHATSUPP-SSH/src/app.js

Hurray! 🙌 your app is now up and running, get started by sending a simple command to your Twilio sandbox over Whatsapp.

What are all the Unique features?

Apart from executing traditional shell commands, our app supports and has cool features built-in. Here they are...

in-app user authentication. Before executing any command, the user has to authenticate himself by entering the correct password. He can continue, only if the authentication is successful.

📌 The execution thread gets locked automatically every 5 minutes(once after the user is authenticated), even if no operations were performed.
The user has to re-enter the correct password to continue...This is to stay safe from non-auth users. The lock interval could be modified in src/utils/validators/authValidator.js

Helper commands. ssh-help is the command to view the list of built-in helper commands.
ssh-history gives the history of remotely executed commands. The log file is saved in logs/requestLogs.log

❗️ Make sure to frequently clear the file. (By setting a cronjob or by doing it manually).

sudo reboot can also be executed, which reboots the system and our node server gets automatically picked up during bootup(as we are using pm2 to manage our node server).

📌 Our node app starts automatically, even if there was a sudden system crash/ Unexpected error occurs.

How secure is it?

It could be explained in different layers...

Layer-1

Webhook Validation. This is technically validating the incoming request and making sure it was sent by Twilio. It is done by verifying x-twilio-signature passed in request headers by Twilio and with different parameters like (authToken, x-twilio-signature, webhookUrl, req.body). More info on this could be found here

Layer-2

in-app authentication. After the request is validated in layer-1, the user needs to enter the correct password to get authenticated and the user's authStatus would be reset every 5 minutes.

Challenges came across

Managing change in directory state. This was very challenging because once the command is executed by the child-process, it gets killed and the further executing process has no idea about the prior change in the working directory.
Custom Authentication. I've talked about it earlier, it was hard to manage the state of the user and to validate each request by also keeping an eye on the last login time.
Error Handling. Needed to take care of different scopes of errors and process/ child-process crashes.
Async code handling. As fs and childProcess modules provide mostly async functions, these needed to be handled carefully.

What I learned along the way?

The development process was just amazing, every day I got to try something new and different. I would say "Learn and ~~Code~~ Explore" had been my mantra throughout the flow. Learned a lot about spawning a new process and dealing with child processes. Got my hands on file-system, custom middlewares, startup scripts/ init.d scripts, systemctl, etc...

Limitations

Currently does not support multiple commands execution at once like... cd Sample && touch hello.txt
Does not know how to react for interactions like when critical commands with sudo are executed.

Link to Code

manojnaidu619 / Whatsupp-SSH

Access your SSH over Whatsapp

Additional Resources/Info

Gmail Login by Automating Browser

Manoj Naidu — Sat, 18 Aug 2018 19:27:09 +0000

Prerequisites

Make sure you run latest version of firefox
Make sure you have installed geckodriver (a firefox webdriver) in your ruby executable path
Run gem install watir to install watir gem

It's show time!!

require 'watir'                       
browser = Watir::Browser.new(:firefox)         # To launch the browser
browser.goto("https://www.gmail.com")
sleep 2
browser.text_field(class: "whsOnd zHQkBf").set "{ YOUR GMAIL ADDRESS HERE }"  
sleep 2
browser.span(index: 4).click
sleep 2
browser.text_field(name: "password").set "{ YOUR GMAIL PASSWORD HERE }"
sleep 2
browser.send_keys :enter

finally, cd to the script path in the terminal and type

ruby FILENAME.rb

Sit back and watch it in action!!

Rails + Ember

Manoj Naidu — Sat, 18 Aug 2018 18:20:13 +0000

Screencast is available Here

Link to my blogspot manojnaidu619.blogspot.com