DEV Community: Quack Quack

I have created a json inspecting tool. What do you think?

Quack Quack — Fri, 26 May 2023 16:38:39 +0000

As a back-end developer, when someone ask me why something doesn't work on the UI or something in particular doesn't show incorrect data on the UI. The first thing i do is to open the Developer Console on the browser and capture the response from the API. Then i inspect that JSON and find some particular field doesn't meet the requirements or the data doesn't meet some specific criteria.

The thing is, inspecting the data using the browser "Ctrl+F" shortcut or copy the whole JSON string then go to VScode or some online json formatter just to find out you have to go through each item in the array of 10 items but has lots of field is very painful. There are other cases where you want to sum a particular field of an array just to get the sum so you could tell if the Front-end guy is calculating the whole thing incorrectly or you are which i find Ctrl+f and json formatter doesn't help at all. So, i created an web app using React js to solve this problem.

Basically, this website let you input the raw json string and do quick filter, mapping, counting on the provided string in a pipeline fashion. Here's some screenshots

Here's the link to the page: https://json-pipe.pages.dev/
This is just the prototype of the project, i will add more feature to make it able to perform complex operators, please tell me what you think and what i should improve 😉.

Anyone using Remixjs for production? Which UI library are you using?

Quack Quack — Fri, 14 Apr 2023 13:42:51 +0000

First thing first, i'm a .NET developer and have some experience with front end stuff but i could only do the part where i create components add logic to it. If you were tell me to create a react app from scratch or webpack, esbuild, bundling stuff, i have no clues.

A couple of months ago. I saw their post on some platform and couldn't remember where it was but the post was quite impressive that they claims their way of fetching data was faster. Lately, i've got an idea and want to spin up a website so i gave it a try but been struggling to get the SSR stuff done correctly since React 18 hydration didn't go very well with Antd and Material UI. Moreover, the example on how to mix Remixjs with MaterialUI is outdated. So, i don't have any clues at the moment how could we do that.

I think we programmer won't lose our job in the next 5 years

Quack Quack — Thu, 02 Mar 2023 07:40:06 +0000

The wild world right now

A wile ago, Microsoft released Github Copilot which is an AI based tool to support people writing code more efficiently using native language.

Recently, the announcement of ChatGPT marked a new era. An era of AI where we simple just ask without doing much.

Follow that, Microsoft released Bing which uses newer data than ChatGPT.

Many creative people tried various things like

Ask ChatGPT to write unit test for their program, teach them to write code...

They even finished an essay within 24 hours with the help of ChatGPT

Now we live in fear of being replace by AI :<

What changed?

From my point of view

We may switch from using Google to use AI platform to learn stuff quicker. (there is some news that Google is releasing their AI, too).
We will learn quicker and many people will join the Programming community.
Speaking of long term, there will be less critical thinking people than ever.

What will remain the same?

First, If we bring the value, we get paid.
From a point of view of "It's just a tool". Remember when people have to write down everything on the paper for accountant then Microsoft Excel came out and people use it as a Tool? Of course, some old people could not catch up and lost their job because the youth are better at learning technology. Nevertheless, they didn't lose their job instantly, they lose their job after a couple of years till all the company shifting from paper to Excel.
People get hired and get paid because they bring the value of accountant done.
From a point of view of "It's gonna replace us who write code". Yes, If AI does a very good job at coding and could make performance and bug-free app then we should give up coding, why would we reinvent the wheel? We should leverage it's skill to make our products or do whatever brings value instead. We use this to bring the value instead of writing code.

What should we do?

Keep learning We are still the one who get the job done on a high level. AI at the moment is just a tool which will makes the job done faster but the decision is still up to us. We should use AI to learn even quicker.
Become a high value person I don't think AI in the next 5 years could write the whole project from scratch base on some vague requirements. In the meantime, we should try to increase our value so the value we bring up could never be fulfilled by the AI.

Conclusion

Keep learning, keep improving yourself to become a high value person and enjoy your life.

Tell me your opinion in the comment section!

Implement base64 encoding using Rust - [Part 2] Handle unicode characters

Quack Quack — Fri, 17 Feb 2023 02:30:33 +0000

Introduction

In my previous post, i implemented base64 encoding but it only works for non-unicode characters. In this post, we will enhance our base64_encode function to make it support the whole planet =)).

What we did wrong?

Let's have a look at RFC4648 and find out what we've been missing the whole time.

The encoding process represents 24-bit groups of input bits as output
strings of 4 encoded characters. Proceeding from left to right, a
24-bit input group is formed by concatenating 3 8-bit input groups.
These 24 bits are then treated as 4 concatenated 6-bit groups, each
of which is translated into a single character in the base 64
alphabet.

And a look at our code from previous post

    for char in input.chars() {
        ...
    }

See the bold section from the quote? that's where we went wrong. In other word, we do not guarantee that our character could be represented using 1 byte, only ASCII character could be represented using 1 byte, but unicode is not the case. For example, 'a pile of poo' 💩 is represented using 4 bytes (F0 9F 92 A9).

What is the fix?

It's easy to fix, we just need to modify the code to loop over each byte in the str. Lucky for us, Rust has a built-in method bytes which create an Iterator to loop over each byte in the str. The good thing about Iterator is that they are lazy and won't do anything until being consumed, we can still keep our memory usage minimal.

    for byte in input.bytes(){

    }

Add more testing

A pile of poo 💩 should be encoded correctly to prove the correctness of our program.

assert_eq!("8J+SqQ==", base64_encode("💩"));

Implement base64 encoding using Rust - [Part 1] Base64 for non-unicode characters

Quack Quack — Wed, 15 Feb 2023 15:04:59 +0000

Introduction

I've been using base64 encoding for a while and haven't got an idea how it works internally so i decided to read some documentation and implement it using Rust.

What will we learn?

Some bit wise and bit shift operations
How base64 works internally?
Why base64 takes up more space?

Implementation

First, let's read up the RFC 4648 to know how things work.

How it works

Basically, the base64 encoding takes a chunk of 3 characters and gives back 4 characters by using the following rule. Each character has 8 bits so 3 character has 3*8 = 24 bit, we simple just split that 24 bits into 4 part (6 bits every part) and encode it using the table below to get 4 characters.

If we don't have enough character to fill up 3 bytes segment. For example, our input is just an "a" which is 1 byte, we will fill up 2 empty byte 0x0 to get the 3 bytes segment. The first 2 characters should be encoded using "a" first 6 left most bits and 2 right most bits following by 2 padding characters "=".

The code

First, we define our alphabet.

static alphabet: [char; 64] = [
    'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S',
    'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l',
    'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '0', '1', '2', '3', '4',
    '5', '6', '7', '8', '9', '+', '/',
];

And also the padding character

static padding_char: char = '=';

We also need to define the signature of our base64_encode function. It will take a &str as input and output a String

fn base64_encoding(input:&str) -> String
{
    todo!()
}

I initially wanted to loop over the string input and create 3 characters segment first, each group contains a pointer to a portion of the input string. Then i would iterate over each group and process them. But i found that it would consume more memory if the input is a very long string. So, i decided to go with one character at the time.

for char in input.chars(){
//do stuff here
}

Before we can process the list, we need to define some rules to process each character.

Each character is a byte, which is 0b00000000. (If you don't know about binary numbers, please spend sometimes to get familiar with the concept Google). So, 3 bytes could be represent like this

0b00000000 0b00000000 0b00000000.

If the character is the first one in the group, then we take it's first 6 leftmost bits and encode it as a character.

0b*000000*00 0b00000000 0b00000000. (take the bold bits)
If the character is the second one in the group, then we take the 2 right most bit of the first character in group and combine it with the 4 left most bits of the second character to get the 2nd encoded character.

0b000000*00* 0b*0000*0000 0b00000000.
If the character is the third one in the group, then we take the 4 right most bits of the second character and combine it with the 2 left most bits of the third characters to get the 3rd encoded character and take the rest for the 4th encoded character. 3rd encoded character should look like this: 0b00000000 0b0000*0000* 0b*000000 and 4th as follow 0b00000000 0b00000000 0b00000000*

Before we can process inside the loop, we will need to create a couple of variables

    let mut result = String::new(); //The result to return
    let mut counter = 0; //the current index in the loop, could only be 0 1 2 represent 1st 2nd 3rd byte in the 3 bytes segment
    let mut left_carry: u8 = 0; //bits left from the previous byte

We will also define the function to handle each character in 3 bytes segment

//we will return the first encoded char and 
fn handle_first_char(first_char: char) -> (char, u8) {
    let char = alphabet[(first_char as u8 >> 2) as usize]; //take the first 6 bits and encode it as a char;
    let carry = ((first_char as u8) & 0x3) as u8; //last 2 bits;

    (char, carry)
}

Next, we write 2 separate functions to handle the second and third character

fn handle_second_char(second_char: char, carry: u8) -> (char, u8) {
    let char = alphabet[(((second_char as u8) >> 4) | (carry << 4)) as usize];
    let carry = (second_char as u8) & 0xF;
    (char, carry)
}

fn handle_third_char(third_char: char, carry: u8) -> (char, char) {
    let first_char = alphabet[((carry << 2) | ((third_char as u8) >> 6)) as usize];
    let second_char = alphabet[((third_char as u8) & 0x3F) as usize];
    (first_char, second_char)
}

Let's go back to our base64_encode function, we will use these handle_first_character, handle_second_character and handle_third_character inside this method.


pub fn base64_encode(input: &str) -> String {
    let mut result = String::new();
    let mut counter = 0;
    let mut left_carry: u8 = 0;

    //return early if the string is empty
    if input.is_empty() {
        return result;
    }

    for char in input.chars() {
        println!("{}", char);
        counter += 1;

        if counter == 1 {
            let (char, carry) = handle_first_char(char);
            result.push(char);
            left_carry = carry;
        }

        if counter == 2 {
            let (char, carry) = handle_second_char(char, left_carry);
            result.push(char);
            left_carry = carry;
        }

        if counter == 3 {
            let (first_char, second_char) = handle_third_char(char, left_carry);

            result.push(first_char);
            result.push(second_char);

            counter = 0; //set counter = 0 to create another 3 byte segment
            left_carry = 0; //reset the carry here
        }
    }

Add padding characters

The algorithm works now, but it's only correct if our string has enough character to fill up 3 bytes segment without any odd characters. If our string has only one character, it would give us wrong result.
Let's add a bit more code to handle padding.

We've mentioned above that a segment is made of 3 characters. If we have 1 character left, we should add 2 more characters as padding and add 1 character in case we have 2 characters left.

//from the padding above
fn calculate_padding(counter: u8) -> u8 {
    //the result could only be 1 or 2
    3 - counter
}

Let's add 2 more functions to handle padding for each case (1 or 2 padding char).

fn handle_padding_first_char(carry: u8) -> char {
    let char = alphabet[(0x00 | (carry << 4)) as usize];
    char
}

fn handle_padding_second_char(carry: u8) -> char {
    let char = alphabet[((carry << 2) | (0x00)) as usize];
    char
}

And modify our base64_encode accordingly

pub fn base64_encode(input: &str) -> String
...
    if counter != 0 {
        let padding = calculate_padding(counter);
        if padding == 2 {
            let char = handle_padding_first_char(left_carry);
            result.push(char);
            result.push(padding_char);
            result.push(padding_char);
        } else {
            let char = handle_padding_second_char(left_carry);
            result.push(char);
            result.push(padding_char);
        }
    }

    println!("{}", result);
    result

Now that's what we call base64 for non-unicode characters.

Testing

Let's write some test as well and use an online base64_encode website to verify that we got the correct answer (You can use this site).

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn it_works() {
        assert_eq!(&base64_encode("crab"), "Y3JhYg==");
        assert_eq!(
            &base64_encode("the brown fox jump over the lazy dog!"),
            "dGhlIGJyb3duIGZveCBqdW1wIG92ZXIgdGhlIGxhenkgZG9nIQ=="
        );
        assert_eq!(&base64_encode(""), "");
        assert_eq!(&base64_encode("a"), "YQ==");
    }
}

Conclusion

You have just learned:

Bit wise and bit shift operations.
How base64 works internally.
The base64 take 3 characters as input then gives back 4 characters as output, that's why it occupies more space than the original string.

As i mentioned above, our implementation only works for non-unicode characters. In the next part, i will show you guys how to do that.

The source code is available on Github here

DEV Community: Quack Quack

I have created a json inspecting tool. What do you think?

Anyone using Remixjs for production? Which UI library are you using?

I think we programmer won't lose our job in the next 5 years

The wild world right now

What changed?

What will remain the same?

What should we do?

Conclusion

Implement base64 encoding using Rust - [Part 2] Handle unicode characters

Introduction

What we did wrong?

What is the fix?

Add more testing

Implement base64 encoding using Rust - [Part 1] Base64 for non-unicode characters

Introduction

Implementation

How it works

The code

Add padding characters

Testing

Conclusion