DEV Community: minkhantwin

How you can utilize the Javascript array method in manipulating data per requirements

minkhantwin — Thu, 27 Oct 2022 16:00:40 +0000

Introduction

As developers, we all have to deal with the array which is filled with real-world data. I believe it's a luxury to know the tips and tricks of the Javascript array method and how we can overcome difficulties to get the desired output.

let's get started

I believe you would already know these array methods described below.

map()
reduce()
filter()
splice()
slice()

I'm gonna analyze each of these methods and discuss what kind of operation we can perform to achieve the desired result(output).

map()

As an iterative method, the map method loops every item of the array and returns a new array populated with the returned values of the callback function from each index.

Usage

Try to take a look at this array cause I'm going to be referencing this array for multiple times.

//let's say Your database or API returns this object array
let users = [
    {
        id: 1,
        name: 'Chris',
        age: 30,
        // etc ....
    },
    {
        id: 2,
        name: 'Mark',
        age: 16,
    },
    {
        id: 3,
        name: 'Alice',
        age: 25,
    }
];

Extracting the specified field of the individual array item

Let's say you need only the names of each user throughout the process. Having only an adequate field of data makes future data processing and debugging more easier and efficient.

let userNames = users.map(p => {
    // where we can return any type of data we like
    return p.name;
    // return `Sir / Madam ${p.name}`;
})
console.log(modifiedUsers);

Modifying array item

Let's say you have to add another field for each user object based on whether the user is an adult (18 or above) or not for another feature(s).

let modifiedUsers = users.map(p => {
    if (age < 18)
    {
        // can change any field of each user object
        p.isAdult = false;
    }
    return p;
})
console.log(modifiedUsers)

filter()

Like every other iterative method, the filter() method loops through the array. Unlike other methods, it does not necessarily return every index of the array item.

Usage

Filtering based on the field value of the array item

Let's say you want a user array who is already an adult

newUsers = users.filter(u => {
    // can make any condition to filer out
    return u.age >= 18;
});
console.log(newUsers);

Indexing and extracting the specified item inside the array

Let's say the user id is known and you need to get the whole object of the user. This kind of scenario occurs all the time at either front-end or back-end development.

let requestedId = 2;
let currentUser = users.filter(u => u.id === requestedId)[0];
console.log('currentUser - ', currentUser);

reduce()

With reduce method, we can calculate the aggregation on the array.

Usage

Concatenating the value of the array items

Let's say you want a string of user names separated by a comma. You can do this:

const tempStr = users.reduce((str, u, i) => {
    str += u.name;
    if (i < users.length)
    {
        str += ',';
    }
    return str;

}, '')
console.log(tempStr);

Finding max and min among the items

Let's say you want the youngest user among all users.

const youngest = users.reduce((xUser, u) => {
    if (xUser === null || xUser.age > u.age)
    {
        xUser = u;
    }
    return xUser;
}, null)
console.log(youngest);

splice()

The splice() method changes the contents of an array by removing or replacing existing items and/or adding new items.

Usage

Take a look at the countries array and let's say the array is ordered by some sort of ranking. Beware that 'Indonesia' is misspelled and a weird 'Unknown' item is present in the array.

const countries = ['Singapore', 'India', 'Indonesai', 'Unknown', 'China'];

Manipulating the placement of the array item

To replace the 3rd item (Indonesia) with the correct spelling of Indonesia,

countries.splice(3, 1, 'Indonesia');
console.log(countries)

You need to add Thailand between Singapore and India.

// add Thailand at index 1
countries.splice(1, 0, 'Thailand')
console.log(countries);

To remove the 'Unknown' item,

const removed = countries.splice(3, 1);
console.log('Removed item - ', removed);
console.log('Modified Countries Array - ', countries);

slice()

The slice() method returns a shallow copy of a portion of an array into a new array object selected from start to end (end not included) but the original array will not be modified.

Usage

Let's say you have this article array.

const articles = [
    {
        title: 'Title 1',
        created: '2022-2-2'   
    },
    {
        title: 'Title 2',
        created: '2022-2-3'   
    },
    {
        title: 'Title 3',
        created: '2022-2-4'
    }
];

Extracting the requested number of the array for pagination

Let's say you want the data for page 1 and each page will have 2 items.

let page = 1, 
    limit = 2;

// for first page
let start = (page - 1) * limit,
    end = start + limit
const firstPage =  articles.slice(start, end);
console.log(firstPage);

Similarly for page 2,

// for second page
page = 2;
start = (page - 1) * limit;
end = start + limit;
const secondpage = articles.slice(start, end)
console.log(secondpage)

Conclusion

Throughout the article, I show only individual usage of each method. You can combine two or more of these methods for the more complex operation to improve your coding skill.

Simplified Kafka concept and example with nodejs

minkhantwin — Mon, 24 Oct 2022 05:19:14 +0000

Introduction

Kafka is a distributed-event store and stream-processing platform supported by Apache.

How it works

Kafka improves the traditional server-client mechanism by putting a layer between server and client which acts as a connector storing the events, produced from the servers(producers) through the specified topic, and serving the event stream to the client(consumer).

Prerequisites

Need to have docker application on your machine
Need to have an intermediate knowledge of how nodejs application works

Let's start coding

In this article, we are gonna use ready-made docker compose file, which setup for kafka service, To create a kafka-node application, follow the steps described below.

Open docker application

Create a new file named docker-compose.yml in your project directory and copy the following codes into the file

version: "3"
services:
zookeeper:
image: 'bitnami/zookeeper:latest'
ports:
  - '2181:2181'
environment:
  - ALLOW_ANONYMOUS_LOGIN=yes
kafka:
image: 'bitnami/kafka:latest'
container_name: 'kafka'
ports:
  - '9092:9092'
environment:
  - KAFKA_BROKER_ID=1
  - KAFKA_LISTENERS=PLAINTEXT://:9092
  - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
  - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
  - ALLOW_PLAINTEXT_LISTENER=yes
depends_on:
  - zookeeper

Run the following command in the directory same as the docker file to start the kafka
```
docker-compose up
```
Run the following commands to initialize the node project
```
npm init
```
Run this to install the node package which will let you connect to kafka server in your node application
```
npm install node-rdkafka
```
For producer and consumer, create index.js files in their respective folder

You can emit events from producer by the following code,

import Kafka from 'node-rdkafka';
console.log('Producer ...');
const stream = Kafka.Producer.createWriteStream({
    "metadata.broker.list": "localhost:9092"
    },
    {},
    { topic: 'test' }
)
function queueMessage()
{
    const message = 'Hello world'
    const success = stream.write(Buffer.from(message))
    if (success)
    {
        console.log('success');
    }
    else 
    {
        console.log('something went wrong!')
    }
}
setInterval(() => {
    queueMessage()
}, 3000)

You can subscribe the specified topic and get the sequence of events streamed from the producers as follow,

console.log('Comsumer ...')
import Kafka from 'node-rdkafka';

const consumer = Kafka.KafkaConsumer({
    "group.id": "kafka",
    "metadata.broker.list": "localhost:9092"
    },
    {}
);

consumer.connect();

consumer.on('ready', () => {
    console.log('consumer ready . . . ')
    consumer.subscribe(['test']);
    consumer.consume();
}).on('data', (data) => {
    const buf = Buffer.from(data.value);
    console.log('Message received: ', buf.toString());
});

We need to create commands in the package.json file to run the producer and consumer.
```
"scripts": {
    "start:producer": "node producer/index.js",
    "start:consumer": "node consumer/index.js"
},
```
Add the above lines inside the scripts block.
Finally, we can run both producer and consumer by enter the following lines into the console.
```
npm run start:producer 
```
```
npm run start:consumer
```

You can see the message emitted by producer inside the consumer console.

Conclusion

Producer can emit events to the kafka server through the topic. Theses events are persistant inside the server and queued like a log file. When consumer make a request with the certain topic, kafka server response all the queued event reside in it back to consumer.

Reminder

Please note that this is the simplified version of how kafka works. Hope you will have the overview of Kafka.