DEV Community: Niharika Singh ⛓

Learn how to create a video summarizer using Whisper from OpenAI and BART from Meta on Hugging Face

Niharika Singh ⛓ — Wed, 13 Sep 2023 14:27:20 +0000

Originally posted on: https://writings.niharika.me/how-to-improve-youtube-with-llms

Last update: Wednesday, September 13, 2023

The way humans interact with technology has come a long way. In the early days of computing, users had to input commands using punch cards. At this point, if you have to pause and consciously think about "punch cards," then I proved my point quite effectively;

Humans have indeed come a long long way in how we interact and navigate the digital world.

The story of the metaverse is shaping up rapidly becoming increasingly natural and intuitive. We are leveraging touch, voice, gestures, and probably direct brain connections in the near future.

Lightning-fast advances in Natural Language Processing (NLP) have enabled us to engineer even better user experiences. Just when you think there is probably no more room left for optimization, there is. NLP effectively makes machines understand and generate human language. This creates a significant impact on how we communicate with technology.

🎯 The purpose of this blog is to explore how YouTube can be improved by capitalizing on the latest groundbreaking advancements in LLMs and to create a video summarizer using Whisper from OpenAI and BART from Meta.

How would I improve YouTube

As a product manager, it is key for me to identify challenge areas in a product and strategically devise solutions that not only address these issues but also align with the larger product vision and goals. The product in question here is YouTube.

User groups

I don't think I need to define what YouTube is. So I can skip this part and get to defining the key user segments of YouTube. Broadly speaking, they fall into two categories:

Creators
Consumers

For the purpose of this blog, I will focus on the consumers. Without consumers, creators lose their value. Therefore, it is key to enhance the consumer's experience at every step of the way.

Potential pain points / challenges

Misleading thumbnails and titles: Clickbaits. Extremely frustrating for viewers seeking relevant content.
Inappropriate content: Despite YouTube's content moderation efforts, viewers may come across content containing offensive, harmful, or inappropriate content.
Content duplication: What is more annoying than different creators uploading the same video repeatedly cluttering the search results and making it difficult to find the original video.
Limited content discovery: If YouTube's recommendation algorithm doesn't align with the viewer's interest, it can be very difficult to discover new and relevant content.
Limited content accessibility: Imagine not being able to understand a video just because you don't know the language it is in. Most of the content by top channels on YouTube is in English.

Ideas that may help improve UX of consumers

Natural language interaction with the video: Viewers can read the text summary of the video before jumping into watching a video. Let's say the video is a 50-minute-long panel discussion. This improvement will also save a lot of time of the viewer, helping them make a more informed choice about the content they will consume. The user can also ask questions and get answers in text without necessarily watching the video.
Detect and flag inappropriate + duplicate content
Enhanced recommendation algorithms
Multilingual support and accessibility: Viewers can view the video in any language they like in the same voice as the narrator. This will enhance the reach of every video bringing down the language barriers.

Time to create a video summarizer

Personally, for me clickbaits are annoying and sometimes before committing to watching a long video, I'd like to judge the video based on a text summary. At the moment, I rely on comments on the videos. However, there are instances when there are not many helpful comments or maybe commenting had been turned off.

I will address this painpoint by leveraging LLMs that will generate a text summary of any YouTube video I give as an input.

In a nutshell...

Web interface

The user will enter the link to the YouTube video and click on 'Summarize Video' button.

Let me enter this YouTube video and see what comes out.

Here's the output:

Summary:

"The original iPod was a pioneer in terms of the simple design and easy-to-use functionality. Apple credited the idea of the iPod to an obscure man by the name of Ken Kramer. The rise of the MP3 file format caused a lot of commotion in the music industry."

Quite good, no?

Here's the python script I'm using to do this magic:

import pytube
import requests
import os
from dotenv import find_dotenv, load_dotenv
import openai
import streamlit as st

st.title('YouTube Video Summarization')

load_dotenv(find_dotenv())
openai.api_key = os.getenv("OPENAI_API_KEY")
HUGGINGFACEHUB_API_TOKEN=os.getenv("HUGGINGFACE_API_TOKEN")

# Extract audio from YouTube
def get_audio(video_url):
    # Create a PyTube object for the video.
    youtube_video = pytube.YouTube(video_url)

    # Get the audio stream from the video.
    audio_stream = youtube_video.streams.filter(only_audio=True)

    # Get title
    st.write("Now summarizing: ", youtube_video.streams[0].title)

    # Download the audio stream to a file.
    audio_stream[0].download(output_path="audios", filename="audio.mp3")

# Audio to text
def get_text(filename):
    audio_file= open(filename, "rb")
    transcript = openai.Audio.transcribe("whisper-1", audio_file)
    return transcript

# Summarize
def summarize(transcript):
    API_URL = "https://api-inference.huggingface.co/models/facebook/bart-large-cnn"
    headers = {"Authorization": f"Bearer {HUGGINGFACEHUB_API_TOKEN}"}

    payload = {
        "inputs": transcript
    }

    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()


youtube_link = st.text_input('YouTube Video:', 'Enter URL here')

def process_input(youtube_link):
    get_audio(youtube_link)
    video_text = get_text("audios/audio.mp3")
    video_text = video_text.text
    summary = summarize(video_text)
    return summary[0]

# Create a button to trigger the function
if st.button("Summarize Video"):
    result = process_input(youtube_link)
    st.write("Summary", result)

Circling back

Injecting LLMs into YouTube to enhance the UX and revolutionize the way we engage with videos on literally any platform is beyond awesome.

By providing video summaries, improving search and discovery, and enhancing accessibility and convenience, YouTube can become an even more user-friendly and informative platform.

As a product manager, I believe that exploring these possibilities and addressing the challenges will lead to a more enriched YouTube experience for everyone. It's time to harness the power of LLMs and take YouTube to the next level of user satisfaction and engagement.

Demystifying Var, Let, and Const

Niharika Singh ⛓ — Sun, 06 Sep 2020 11:12:51 +0000

I know that a lot of devs already know the difference between var, let, and const. However, it is also true that many devs in the beginning of their learning journey might not know about the differences.

So...

There are three ways of declaring variables in JavaScript:

var
let
const

Today, most relevant browsers support let, var, and const. let keyword is still not supported in Opera Mini.

let keyword is still not supported in Opera Mini.

Let's get started with `var`

var was the primary way to declare variables until 2015. In ES6, the concept of let and const were introduced.

var is scoped to 'current execution context' i.e. it is scoped to the function enclosing the variable or the global scope.

var x = "hello world";

function hello() {
    var x  = "hey there"; // redeclared
    x = "hola!"; // reassigned
    console.log(x); // OUTPUT: hola!
}
console.log(x); // OUTPUT: hello world
hello();

This tells that variables declared using var keyword can be reassigned as well as redeclared to a different value.

`let` is almost similar to `var`

let is block scoped. NOT context scope. This means that a variable declared with let is only valid within { }.

let colour = "red"; 

function hello() {
    let colour = "blue"; //different variable
    colour = "pink"; // can reassign
    let colour = "orange"; // can't redeclare
    console.log(colour);
}
console.log(colour); // OUTPUT: red
hello();

error: unknown: Identifier 'colour' has already been declared (6:8)

  4 |     let colour = "blue"; //different variable
  5 |     colour = "pink"; // can reassign
> 6 |     let colour = "orange"; // can't redeclare
    |         ^
  7 |     console.log(colour); // OUTPUT: pink
  8 | }
  9 | console.log(colour); // OUTPUT: red

This tells that let is scoped within a { } block. It can be reassigned but can't be redeclared in the same scope. The variable colour outside hello() and inside hello() are two different variables.

Let's see what happens if we try to access a let variable outside the { }.

for (let i = 0; i < 5; i++) {
  console.log(i);
}
console.log(i);

0
1
2
3
4
error: Uncaught ReferenceError: i is not defined

This means that i is not defined beyond { }.

`Const`

Const stands for constant. Like let, const is also scoped to a block. Like let, const can't be redeclared within the same block scope.

The idea is that the reference of the const variable can't change. This doesn't mean that the variable is immutable. It just means the memory reference can't change.

const x = 5;
x = 10; // error: Uncaught TypeError: Assignment to constant variable.

const y = ["a", "b", "c"];
y.push("d"); // array can be manipulated
console.log(y); // OUTPUT: ["a", "b", "c", "d"]

Array declared with const keyword can be manipulated because the memory reference the variable y is NOT changing. Const doesn't care about the value of the variable. It only cares about the variable reference.

Unlike let and const, you can't initialise a const variable without a value.

var x; // OK
let y; // OK
const z; // NOT OK

error: unknown: Unexpected token (3:7)

  1 | var x; 
  2 | let y;
> 3 | const z;
    |        ^

This pretty much sums up var, let, and const. One thing I didn't touch upon in this post is variable hoisting.
I think it would be a better idea to break it up in two parts. So in the next part, we will look at var, let, and const from variable hoisting point of view.

Cheers!

10 Things To Know As A Fullstack Developer 👁️👄👁️

Niharika Singh ⛓ — Wed, 02 Sep 2020 10:46:35 +0000

Being a fullstack developer is hard. What's even harder is to keep up with the ever changing dynamics of development.

In this blog, I have outlined 10 most important things every fullstack dev should keep in mind while honing their skills.

This post is ideally for a developer who is just starting their journey to become an AWESOME fullstack dev.

1. Frontend first

Mastering client side (frontend) before server side (backend) is a good strategy. You will see your actions getting manifested in real time on the browser. This will keep you motivated to build more.

2. Fundamentals

It's VERY important to know the fundamentals of any framework by heart. If you claim to know the deeper intricacies of a framework and are unsure about the basics, then you need to revisit your learnings and probably also learning style. In a coding interview, this will be seen in a SUPER negative light. You'd want to avoid that.

3. 5 hours of JS and 5 years of JS

If you don't know JavaScript yet, then it will take you a few hours to know it enough to get started with basic web development. However, to understand the subtle nuances of JavaScript, it will take years of practice. Get ready for this kind of a commitment.

4. Basic web security is mandatory

Fundamental knowledge about web security is expected from a fullstack dev. You should know the core of cryptography, HTTPS, application security, Virtual Private Networks (VPN), Firewalls, SSH, etc.

💡 TIP: Ask yourself if you'd really use what you've build if it was a real service online. If the answer is no, then the application needs improvement.

5. SQL is as important as NoSQL

Do not think NoSQL is better than SQL or vice versa. Know when to use which database. To be a good fullstack dev, you should know both.

6. Side projects 🔥

The best way to learn is by doing it. Try to build about 10 side projects. Add them to your portfolio. Cloning already existing services is a good way to learn. It will save you a lot of planning time. You can create a YouTube / TikTok / Instagram clone.

7. Host them projects

A rookie mistake you could make is to not host your project. Make sure you host it. Heroku is a good place to start. If your project isn't hosted, its as good as never made.

8. CI/CD will make you stand out ⭐️

To shine in a tech interview, make sure you know your CI/CD well. Have a structured GitHub repository and ensure CI/CD is working properly. Have proper unit tests in place. This will not only prove your competency in CI/CD but also show that you ship quality code and nothing else.

If you're new to CI/CD, I'd recommend you to check out GitHub actions to create a CI/CD pipeline.

9. Basic DevOps is mandatory

Nginx, load balancing, docker, caching is expected. There are no two thoughts about it.

I've noticed that people who are truly in love with computer science have a knack for DevOps. 😉

10. Get comfortable with cloud ☁️

If you are a cloud virgin, it can be daunting at first. But cloud is an acquired taste. Once you get into the groove, you will fall in love with it.

Check out the services offered. If they fit well with your project, consider integrating them. They will fetch you brownie points.

BONUS: Smart goal setting

Create a full fledged plan for your learning. I personally use Notion for this purpose. Make sure all the tasks are clearly defined so that you don't have to waste time planning it out while you're building. This can save you a lot of time.

I hope this blog helped you align your learning goals.

Cheers!

👁️👄👁️

Why should I learn JavaScript

Niharika Singh ⛓ — Thu, 27 Aug 2020 07:01:19 +0000

There exists an extreme relationship between JavaScript and JavaScript developers.

LOVE-HATE RELATIONSHIP.

Sounds like a freakin’ dramatic roller coaster.

Why are you still in this relationship? What makes you stay?

Basically, why should I learn JavaScript?

(Wrong answers are also welcome haha)

Which CI/CD tool do you use?

Niharika Singh ⛓ — Wed, 26 Aug 2020 07:43:54 +0000

I have been researching a lot on CI/CD tools lately.

I have worked on TravisCI before, and recently with GitHub Actions.

I'd love to know which ones do you use and why!

Get These In Place Before Starting Your Next Side Project 👀

Niharika Singh ⛓ — Tue, 25 Aug 2020 07:55:09 +0000

Let me introduce you to something that you might already be familiar with— Side Project Hell 😈

Basically, you end up with like 747324856 side projects and not a single one is really complete.

I believe, this hell would be most common in the JavaScript world. Every other day you get a kickass library and everyone wants to try it out by building their own something.

To save yourself from this horror, set the following items in place:

⚠️ This is just a list, no magic. If you still F it up, can't help you there. 🤷‍♀️

1. Announce your idea

This is a bold step, but super effective. Tell the world you have started working on a side project. Describe it. Tell them by when will they be able to see/use it.

Most likely, you will get a supporting response. This will motivate you to continue building whenever you feel like abandoning it.

2. Get a partner

Find yourself a code buddy who complements your skills. If you're a good webdev, find yourself a good devops person.

This will accelerate the building process and learning will be symbiotic.

3. Don't start coding right away! Align your vision.

Many people mess it up by starting to code too early.

First of all, put in place all the requirements. This could be in terms of features, or learning goals. Divide these features/modules in phases. In the first phase, keep it minimalistic.

4. No CSS in initial phases

Do not think about beautifying the project in this phase. It will distract you from your actual goal.

You will end up spending crazy amounts of time on HTMLing and CSSing.

Set the building foundation first, then the building, then the paint.

5. Find a mentor

If possible, get yourself a mentor who can guide you throughout your learning journey.

This could be your college senior, or a friend who has been working since a while.

6. Set up Git

It is SUPER FKN essential to commit at every milestone. No matter how small.

Make sure you write good commit messages. Simply committing with fix or update or finally is bad practice.

We all do it, but should not.

7. Set up CI/CD pipeline

This will make you shine among others. You will learn a hell lot.

Even the tiniest of projects deserve CI/CD. You will fall in love with it eventually. It's more of an acquired taste. ;)

8. Stay hungry, stay foolish!

Make sure you're always on a lookout for feedback. Take it seriously.

You should also give feedback to people in your network.

At the end of the day, don't forget what you really want. If you want to complete that side project, do it. There's no better time than now.

Cheers!

Unravelling JavaScript Console API

Niharika Singh ⛓ — Mon, 24 Aug 2020 11:49:35 +0000

Nights that end with console.log() usually precede mornings that start with console.log(). 😔

Well, there's life beyond console.log().

Let's go over some JavaScript console methods that REALLY have the potential to make your life better.

0. `console.log()`

console.log("You already know this!");

// This is used to output a message to the console.

Output:

1. `console.table()`

Sometimes we want to output a large object on the console. Using console.log() can be a pain.

console.table() is a treat to sore eyes.

function Crypto(symbol, name) {
  this.symbol = symbol;
  this.name = name;
}

let bitcoin = new Crypto("$BTC", "Bitcoin");
let ethereum = new Crypto("$ETH", "Ethereum");
let polkadot = new Crypto("$DOT", "Polkadot");

console.table([bitcoin, ethereum, polkadot]);

Output:

How beautiful and incredibly readable does that look!

If you had used console.log() here, this is what you would have seen instead:

2. `console.info()` / `console.error()` / `console.warn()`

The console.info() method outputs an informational message.

The console.error() method outputs an error message.

The console.warn() method outputs a warning message.

console.info("FYI: Today is Monday");

console.error("Today is not Friday");

console.warn("Get back to work!");

Output:

💡 It is a good practice to log messages on the console with appropriate formatting.

3. `console.time()`

This API method will tell you runtime in milliseconds. You can have multiple timers running at once in a JS program. (Maximum 10,000 timers.)

console.time("time taken");
//DO SOMETHING
let x = 90;
let y = 100;
let z = x + y / 100;
console.timeEnd("time taken");

Output:

time taken: 0ms - timer ended

These were some lesser known methods of console API in JavaScript.

I am hopeful that these methods will help you!

Cheers.

How To Master A Programming Language Really Fast 🔥

Niharika Singh ⛓ — Sun, 23 Aug 2020 10:36:18 +0000

Having competency in at least one programming language is SUPER important for a developer. Developing competency in more than one language gives you an edge.

You might know some people who can pick up languages in no time.

When you know a language really well, you might think that the only thing that will change with other languages is the syntax.

You can't be more wrong.

The only thing that changes when you move from JavaScript to Rust is NOT the syntax.

Programming style also changes. Key fundamentals change.

A couple months ago, I was learning to program in Rust by using the docs. (This is a really good resource for anyone who wants to get started with Rust)

In my personal experience, learning curve of Rust lang 🦀 is really steep. No amount of books/documentation can conclude your learning.

Luckily, around this time I was solving a lot of competitive programming questions on LeetCode.

I decided that I'll take up a 30 day challenge: Solve 30 programming questions in Rust.

THAT really made all the difference.

Let's take Contiguous Array as an example question.

Given a binary array, find the maximum length of a contiguous subarray with equal number of 0 and 1.

Example 1:

Input: [0,1]
Output: 2
Explanation: [0, 1] is the longest contiguous subarray with equal number of 0 and 1.

Example 2:

Input: [0,1,0]
Output: 2
Explanation: [0, 1] (or [1, 0]) is a longest contiguous subarray with equal number of 0 and 1.

Note: The length of the given binary array will not exceed 50,000.

Solving it in Java was quite simple:

class Solution {
    public int findMaxLength(int[] nums) {
        int sum = 0;
        int res = 0;
        // build a <sum, index> hashmap
        Map<Integer, Integer> map = new HashMap<>();
        map.put(0, -1);
        for (int i = 0; i < nums.length; i++) {
            System.out.println(i);  
            if (nums[i] == 0) {
                sum++;
            } else {
                sum--;
            }
            if (map.containsKey(sum)) {
                res = Math.max(res, i - map.get(sum));                
            } else {
                map.put(sum, i);
            }
        }
        return res;
    }
}

Also, in JavaScript it is simple:

/**
 * @param {number[]} nums
 * @return {number}
 */
var findMaxLength = function(nums) {
    let hash = {0:-1};
    let count = 0;
    let max = 0;
    for (let i=0;i<nums.length;i++) {
        if (nums[i] == 0) count--;
        else count++;

        if (hash[count]!=null) max = Math.max(max, i - hash[count]);
        else hash[count] = i 
    }
    return max;
};

But the real challenge was to solve it in Rust. You search and then you research until you finally learn.

use std::cmp;
use std::collections::HashMap;

impl Solution {
    pub fn find_max_length(nums: Vec<i32>) -> i32 {
        let mut sum: i32 = 0;
        let mut len: i32 = 0;

        let mut map = HashMap::new();
        map.insert(0, -1);

        let size = &nums.len();

        for i in 0..*size {
            if nums[i as usize] == 0 {
                sum -= 1;
            } else {
                sum += 1;
            }
            if map.contains_key(&sum) {
                let x: i32 = match map.get(&sum) {
                    Some(&s) => s,
                    None => -1
                };  
                len = cmp::max(len, (i as i32 -x));
            } else {
                map.insert(sum, i as i32);
            }
        }
        len
    }
}

In short, start solving competitive programming questions in the language you want to learn. You will see a stark difference within a month!

Let me know your thoughts in discussion below.

Cheers!

Productivity Hacks To Get Things Done. No BS.

Niharika Singh ⛓ — Sat, 22 Aug 2020 11:10:15 +0000

Procrastination. Ugh. Can't live with it, can't live without it.

"There's so much to do, and so little time."

"Man, how tf am I going to finish this by then!"

"God..."

"Fu..."

BEEN THERE, DONE THAT?

Let's kill the sloth in you.

Try out these tips/tricks, and let me know which ones worked for you!

1. Batch processing

This is especially helpful for mundane tasks. Batch similar tasks together to operate at greater efficiency.

Batch your phone calls. Batch your emails. Batch your bills.

DO NOT CHECK YOUR EMAIL 10 TIMES A DAY! This is a sin.

2. Love your workspace

You should like to sit and work in your workspace. If something about it ticks you off, change it. Keep it clean and organised.

If you don't love your workspace, what are you even doing...

3. If a task takes 2 minutes or less, do it now.

I know a lot of developer friends who preempt using the washroom to save 80 seconds. NO KIDDING. Stop with that madness.

4. Prioritise!

One of the most crucial decisions you have to make while prioritising your task is to think if you should first do important things or urgent things.

💡 TIP: Important things first.

Most people default to doing urgent tasks first — the problem is that if you're always doing urgent tasks, you're not able to accomplish important tasks. Urgent tasks always come up. And there will always be more urgent tasks than you have time to accomplish, no matter how hard you try. (Source here)

5. Divide and conquer

Break down large or complex tasks into small chunks. This way you will have multiple small wins throughout the day. To stay productive, it is necessary to have a healthy amount of dopamine in your system.

6. Brainwave symphonies

Personally, I love listening to some Chopin while working. I know many people who also like to listen to lofi. The music will help you focus your energy.

7. Figure out when are you most productive

You could be highly productive early in the morning or late in the night or even in the afternoon. Figure out when is your body more alert, and capitalise on that.

💡 TIP: Don't schedule meetings or meet people at this time. This time is super precious for you.

8. Templatize everything

I use Notion a lot. I have a template for every damn thing in my life. This helps me reduce number of decisions I make everyday. Believe it or not, decision fatigue is a real thing!

Just for giggles: Why do you think managers are always stressed out without doing anything really impactful? 😉

9. The famous 80/20 rule ~ Pareto principle

80% of results come from 20% efforts.

Understand that being productive does not mean to do more work. It is about generating more value in less time.

Carefully analyse where are you spending your 24 hours at, and then optimise. Automate 80% of the efforts and then improve the remaining 20%.

10. Eat healthy, live healthy, drink well.

Can't stress enough on this point.

Stay peachy!

6 Awesome Ways To Present Your Code 🔥

Niharika Singh ⛓ — Fri, 21 Aug 2020 10:49:11 +0000

At some point, every developer has to present code snippets. This could be for a presentation, or a tweet, or a blog, or just to go through a snippet later in time.

Let's dive into the tools you can use to present your code awesomely.🔥

1. carbon.now.sh

Create and share beautiful images of your source code.
Start typing or drop a file into the text area to get started.

You can also connect GitHub with carbon.

2. codeimg.io

This is my personal favourite. ❤️️

The best part about codeimg is that they have already existing Social Media Templates. So you don't have to worry about canvas sizing.

Feature wise, you get more options in codeimg as compared with carbon.now.sh.

Here's a Twitter template created with codeimg.io

3. Pastie

If you like minimalism and simply want to share code as a link instead of a screenshot, Pastie is for you.

This is perfect to share code among friends/colleagues. The link generated with Pastie is valid for 24 hours.

CAUTION: Pastie uses HTTP and not HTTPS. This means that it lacks encryption and is not secure.

4. Rust playground

If you are a Rust person, this one is for you.

Play around with browser interface to the Rust compiler to experiment with the language. It also has the capability to share snippet link just like Pastie.

5. Silicon

This is carbon.now.sh implemented in Rust language.

Problems with carbon.now.sh that are overcome by Silicon:

Cannot work without Internet & browser.
Doesn't work well with shell. (Although there is carbon-now-cli, its experience is not very good, especially when the network is not so good.)

Usage:



$ silicon main.rs -o main.png

6. Polacode

If you love VS Code, this one is for you! Polacode is a VS Code extension that takes beautiful screenshots of your code right from VSCode.

The best part about these 6 tools is that they are all free to use!

Cheers.✌️

Language Check

Niharika Singh ⛓ — Sun, 16 Aug 2020 14:02:40 +0000

My Workflow

Language Check action automatically filters issues and pull requests that contains objectionable language (profane words) and instantly reports the action URL on your chosen Slack channel.

This workflow is a must-have for all the developers out there who contribute to open source projects and may get trashed for their hard work 🤦🏻‍♀️

Maintaining sanity in the open source community is essential AF. 😉

This workflow is triggered each time there is an issue is opened, or there is a new comment on an existing issue, or there is a pull request.

⚠️ To test this flow by yourself, you need to have a #blockchain channel in your workspace. Or, you can edit the yml file and change SLACK_CHANNEL there.

⚠️ You also need to generate SLACK_WEBHOOK token for your own Slack workspace and save it in GitHub secrets. To set it up, see this. The slack channel I am using: degen-gh-actions.slack.com. So, the token for this workspace would be different than yours.

Submission Category:

Maintainer Must-Haves

Yaml File

name: Language Check
on:
  issues:
    types:
      - opened
      - edited
  issue_comment:
    types:
      - created
      - edited
  pull_request_review_comment:
    types:
      - created
      - edited
jobs:
  echo_issue_comment:
    runs-on: ubuntu-latest
    name: profanity check
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Profanity check step
        id: set_output
        uses: niharrs/mind-your-language-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      - name: Slack Notification
        if: steps.set_output.outputs.result == 'bad'
        uses: rtCamp/action-slack-notify@master
        env:
          SLACK_CHANNEL: blockchain
          SLACK_USERNAME: moralPolice
          SLACK_COLOR: red
          SLACK_MESSAGE: ':warning: Profanity found on repository'
          SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

Link to Code

niharrs / language-check

Hackathon Entry for GitHub Actions Hackathon on DEV

Submission Category: Maintainer Must-Have

Description of the workflow

Language Check action automatically filters issues and pull requests that contains objectionable language (profane words) and instantly reports the action URL on your chosen Slack channel.

This workflow is a must-have for all the developers out there who contribute to open source projects and may get trashed for their hard work 🤦🏻‍♀️

Maintaining sanity in the open source community is essential AF. 😉

This workflow is triggered each time there is an issue is opened, or there is a new comment on an existing issue, or there is a pull request.

This workflow makes use of two already existing GitHub actions:

Mind your language action: I forked it and tweaked the entrypoint.sh to use conditional jobs and workflows offered by GitHub Actions API.
Slack Notify - GitHub Action: This action is highly…

View on GitHub

Additional Resources / Info

This workflow makes use of two already existing GitHub actions:

Mind your language action: I forked it and tweaked the entrypoint.sh to use conditional jobs and workflows offered by GitHub Actions API.
Slack Notify - GitHub Action: This action is highly customizable, you can send in parameters in the form of environment variables.

Feel free to reach out in case of any issue.

Cheers.✌️

(Part 1) Simplified: Hierarchical Deterministic (HD) Wallets

Niharika Singh ⛓ — Sat, 18 Jul 2020 07:45:17 +0000

If you want to know how Hierarchical Deterministic (HD) wallets work, you're in the right place. We will get down to the fundamentals of cryptography and mathematics to solidify our learning.

To make the most out of this blog, I am assuming you have had some sort of interaction with blockchain in the past. Especially with the process of signing up for a wallet.

So as you may know, each time we sign up for a blockchain wallet, we are presented with a long list of words (sequence-sensitive) that is to be kept a secret. Because if you don't, anyone with this list can withdraw assets from your account.

You can tell if it is an HD Wallet if it presents you with this list of words, formally known as a mnemonic phrase. HD Wallets can potentially have infinite number of account addresses and private keys. Your wallet's UI may choose to present just 10 or 20 of them.

It is important to securely store this mnemonic phrase because this phrase has the ability to regenerate the exact same addresses and their corresponding private keys. And, keep in mind that anybody with this mnemonic phrase or private key will have full control over the entire wallet or the particular address respectively.

In Short...

HD wallets are hierarchically determined from a single seed and you can regenerate the same list of accounts every time using the same mnemonic phrase. The ability to regenerate the same future state using a given set of inputs is called deterministic. Thus, HD wallets are a combination of hierarchical and deterministic.

At this point, you might have some burning questions like: But where do these addresses come from? How does a mnemonic phrase work? Can we pick random words from a dictionary and create our very own mnemonic?

This post will explain the fundamentals of mnemonics by:

Creating a private key
Computing a mnemonic seed using a given private key
Encoding the seed into human-readable mnemonic phrase

I hope that by the end of this article you will understand the nuts and bolts of mnemonic phrases.

“Any sufficiently advanced technology is indistinguishable from magic.” - Arthur C. Clarke

Mnemonic Phrases and Blockchain

The genesis of mnemonics dates back to ancient Greek times. It is a simple memorization technique that encodes and recalls information effectively. In computer science it is both time and memory efficient.

Here’s a well-known mnemonic which helped many of us during elementary math classes; King Henry Died Mother Didn’t Cry Much. This decodes to: Kilometer, Hectometer, Decameter, Meter, Decimeter, Centimeter, and Millimeter. (Yes, the metric system is confusing for some of us.)

Blockchain uses the same technique!

A typical blockchain mnemonic phrase looks like this:

section canal ice eternal city bamboo sunset skill note scare entire couple van ancient absurd window grunt arm runway season found

These words may sound random to an average person, but blockchain wallet software can derive a ton of information from it, like addresses and private keys. This means that your mnemonic phrase is the key to your blockchain accounts. Hence, it is necessary to protect it.

Mnemonic phrases won the confidence of the blockchain community because:

They make account backup very simple by completely eliminating the need to store private keys
The same mnemonic phrase can be used in different wallet software provided by different vendors (given that the same standards are used to implement it)
They enable its users to operate with more freedom. Users can create multiple accounts for diverse purposes. For example, one account for just receiving payments and other one for encrypting files.

Mnemonics in action

Let’s see the mnemonic magic in action by using one of the most badass tools celebrated by the blockchain community created by Ian Coleman; Mnemonic Code Converter (otherwise known as the BIP39 tool). This tool is most commonly used to derive blockchain addresses using a mnemonic phrase.

⚠️ We highly recommend you to download the tool and use it offline. This tool should not be used online. There are countless number of online scams attempting to steal your mnemonic phrase. ⚠️

To use the Mnemonic Code Converter offline, download the bip39-standalone.html file from here. Open this HTML file in any browser. To stay on the safe side, make sure you are not connected to the internet while using it.

You can either create your mnemonic phrase by clicking on the ‘Generate’ button. You can also define the number of words the mnemonic phrase should have. A longer mnemonic phrase guarantees more security because it has a higher degree of randomness or entropy.

To add an extra layer of security, you can enter a mnemonic passphrase. This passphrase is like a lock on your accounts. In other words, a mnemonic passphrase is used to add Two-Factor Authentication (2FA). To unlock the suite of HD accounts, you need to enter this passphrase. Ultimately, this mnemonic passphrase gets encoded in the resultant mnemonic seed. Therefore, having the protection of a mnemonic passphrase enhances entropy of mnemonic phrase. The same mnemonic with different passphrases will yield different accounts.

The BIP-39 tool can not only be used for Bitcoin but for many other coins. For example ethereum, dogecoin, zcash, etc.

You can select the coin for which you want to generate HD accounts. By default, it is set to Bitcoin. If you change the cointype, the addresses generated will be for the coin you selected. The same mnemonic will generate different addresses for different coins. This is simply because the protocol to generate an address from a private key is different for each coin.

Deriving Addresses

On scrolling down, you will see addresses and their corresponding public-private key pair.

All the addresses and keypairs are generated from a single seed (i.e. the mnemonic phrase). This is the beauty of HD accounts.

All accounts created are hierarchically determined from a single seed and you can regenerate the same list of accounts every time using the same mnemonic phrase.

Scientifically speaking, the ability to regenerate the same future state using a given set of inputs is called deterministic.

Thus, HD accounts are a combination of hierarchical and deterministic.

Now, it’s time to put on our first principle goggles understand this process bottom up

In raw form, a mnemonic is nothing but a randomly chosen large number. This number can be 128 bits, 160 bits, 192 bits, 224 bits, or 256 bits. The word ‘randomly’ is essential here because we do not want to create a predictable mnemonic (as this would lead to poor security).

Now, let’s take an unbiased coin. (When tossing an unbiased coin, the probability of heads showing up is the same as that of tails showing up i.e. 0.5.) The next step is to flip it 128 times. Let’s denote heads with 1 and tails with 0. Ultimately we will end up with a 128-bit binary number which may look like:

10101100 11011100 00111110 10010010 01111001 01000000 10100110 01000101 11000101 00011101 00000101 10011101 11000100 01110110 10100000 11101100

In hexadecimal representation, this number is:

acdc3e927940a645c51d059dc476a0ec

The next step is to apply the SHA256 algorithm on the 128-bit random number. This will give us a 256-bit hash.

a5f1cc6ff28228b130455a7eb05f5367f53c9bce9443393e35cbf5a9f8f9a570

The next step is to find the length of the checksum. A checksum is a sequence of numbers or characters. It is used to check any data for errors that may occur due to transmission or storage.

Length of checksum = mnemonic length (bits) / 32

In our case, the mnemonic length is 128. So, the length of our checksum is 128 / 32 = 4.

Now take the first four bits of the 256-bit long hash (should be equal to the length of the checksum) and append these four bits to the end of the randomly generated 128-bit number i.e. a5f1.

In binary, this number is:

10101100 11011100 00111110 10010010 01111001 01000000 10100110 01000101 11000101 00011101 00000101 10011101 11000100 01110110 101000001 11011001 01001011 1110001

Now, make groups of 11 bits each. You will end up with 12 groups. (The total number of bits is 132) Each group represents a decimal index number of a lookup table. The total number of words in this lookup table is 2^11 (2048) with indices ranging from 0 to 2047. The total number of 12-word mnemonics that can be generated is 2048^12.

This lookup table (formally known as a word-list) can be found here. The order of words should not be changed because each word is hard-linked to an index value.

This word-list exists for English, Japanese, Korean, Spanish, Chinese (Simplified), Chinese (Traditional), French, and Italian.

These words make up a human-readable mnemonic phrase.

So now we know how to create a human-readable mnemonic from a random number.

This algorithm is the BIP-39 standard.

We learned how to:

Create a completely random private key by flipping a coin
Compute a mnemonic seed using a given private key
Encode the seed into human-readable mnemonic phrase

While mnemonic phrases were meant to make complicated cryptography human-readable, a list of 12 random words isn’t exactly intuitive for most users to remember or store.

For mass adoption of blockchain technology, we need to create tools that are not just human-readable, they need to be intuitive. There is little meaning in pursuing Layer 2 solutions and ZKSnarks if there isn’t a dead simple fundamental layer comprised of tools that are easy for everyday people to use and understand.

DEV Community: Niharika Singh ⛓

Learn how to create a video summarizer using Whisper from OpenAI and BART from Meta on Hugging Face

How would I improve YouTube

User groups

Potential pain points / challenges

Ideas that may help improve UX of consumers

Time to create a video summarizer

In a nutshell...

Web interface

Circling back

Demystifying Var, Let, and Const

So...

Let's get started with var

let is almost similar to var

Const

10 Things To Know As A Fullstack Developer 👁️👄👁️

1. Frontend first

2. Fundamentals

3. 5 hours of JS and 5 years of JS

4. Basic web security is mandatory

5. SQL is as important as NoSQL

6. Side projects 🔥

7. Host them projects

8. CI/CD will make you stand out ⭐️

9. Basic DevOps is mandatory

10. Get comfortable with cloud ☁️

BONUS: Smart goal setting

Why should I learn JavaScript

Which CI/CD tool do you use?

Get These In Place Before Starting Your Next Side Project 👀

To save yourself from this horror, set the following items in place:

1. Announce your idea

2. Get a partner

3. Don't start coding right away! Align your vision.

4. No CSS in initial phases

5. Find a mentor

6. Set up Git

7. Set up CI/CD pipeline

8. Stay hungry, stay foolish!

Unravelling JavaScript Console API

0. console.log()

1. console.table()

2. console.info() / console.error() / console.warn()

3. console.time()

How To Master A Programming Language Really Fast 🔥

When you know a language really well, you might think that the only thing that will change with other languages is the syntax.

You can't be more wrong.

Productivity Hacks To Get Things Done. No BS.

BEEN THERE, DONE THAT?

Let's kill the sloth in you.

1. Batch processing

2. Love your workspace

3. If a task takes 2 minutes or less, do it now.

4. Prioritise!

5. Divide and conquer

6. Brainwave symphonies

7. Figure out when are you most productive

8. Templatize everything

9. The famous 80/20 rule ~ Pareto principle

10. Eat healthy, live healthy, drink well.

6 Awesome Ways To Present Your Code 🔥

Let's dive into the tools you can use to present your code awesomely.🔥

1. carbon.now.sh

2. codeimg.io

3. Pastie

4. Rust playground

5. Silicon

6. Polacode

The best part about these 6 tools is that they are all free to use!

Language Check

My Workflow

Submission Category:

Yaml File

Link to Code

niharrs / language-check

Hackathon Entry for GitHub Actions Hackathon on DEV

Description of the workflow

Additional Resources / Info

(Part 1) Simplified: Hierarchical Deterministic (HD) Wallets

In Short...

Let's get started with `var`

`let` is almost similar to `var`

`Const`

0. `console.log()`

1. `console.table()`

2. `console.info()` / `console.error()` / `console.warn()`

3. `console.time()`