DEV Community: Frastyawan Nym

Create Local MySQL using Docker Run

Frastyawan Nym — Fri, 22 Mar 2024 02:39:19 +0000

This is just a rant writing. Because how difficult tutorial out there just to setup this kind of thing. So I'm here to summarize for you (reader) how to get it done.

Simple! Run:

docker run --name mysql-local -p 3306:3306 -e MYSQL_ROOT_PASSWORD=admin -d mysql

mysql-local: Your container name, feel free to change it.
-p 3306:3306: Expose container to localhost, so we can connect with UI (ex: DBeaver).
-e MYSQL_ROOT_PASSWORD=admin: root user password, adjust with your preference.
-d: Detach mode
mysql: Image name

This will automatically pull the image if not pulled yet.

Connecting into interface (or else)

After this, maybe you need to access via UI. Here is the setting:

Host: localhost
Port: 3306
User: root
Password: admin

Maybe you will need this

When connecting, maybe you will get error Public Key Retrieval is not allowed. If you are using DBeaver, search for driver properties/setting. Change allowPublicKeyRetrieval = true

Closing

That it and Thank You.

Boost Your Golang Skills: Writing Large Data Files for Performance

Frastyawan Nym — Fri, 15 Sep 2023 06:46:47 +0000

Golang, known for its simplicity and performance, offers various ways to tackle tasks efficiently. As a beginner developer, you might be wondering how to write large data into files effectively. In this beginner-friendly guide, we'll explore three key methods: Sequential, Parallel, and Parallel Chunk. We'll break down the concepts, provide easy-to-understand code samples, and offer practical advice to help you get started. Let's embark on this coding journey! 📝

Introduction 🎉

Before we dive into these methods, let's start with some basics. File handling in Golang is essential for tasks like saving user data, generating reports, or storing logs. Imagine it as managing a virtual filing cabinet where you can add, retrieve, and organize information.

Basic Concepts 🧠

Golang, with its goroutines (lightweight threads), allows you to perform tasks concurrently, making it perfect for efficient file writing. Why does efficient file writing matter? Well, it can significantly impact the performance of your applications, ensuring they run smoothly.

Now, let's explore the three methods with simplified explanations, easy-to-follow code, and practical examples.

Sequential File Writer 📜

What is Sequential Writing?
Sequential file writing is like writing a story from start to finish, one sentence at a time. It's straightforward and easy to understand.

When to Use it?

Use sequential writing when you're just starting, and simplicity is crucial.
Ideal for smaller datasets that don't require super-fast write speeds.
Great for scenarios where data order is essential.

Pros:

Simplicity: It's easy to implement, perfect for beginners.
Data Integrity: Maintains the order of data, making it reliable.
Low Resource Usage: Doesn't require much memory.

Cons:

Slower for large datasets.
Not the best choice for blazing-fast writes.

Code Sample and Explanation:
filewriter/sequential/sequential.go

package sequential

import (
    "fmt"

    "github.com/frasnym/go-file-writer-example/filewriter"
)

// SequentialFileWriter represents a writer for sequential file operations.
type sequentialFileWriter struct {
    fileWriter filewriter.FileWriter
}

// NewSequentialFileWriter creates a new instance of SequentialFileWriter.
func NewSequentialFileWriter(fileWriter filewriter.FileWriter) filewriter.Writer {
    return &sequentialFileWriter{
        fileWriter: fileWriter,
    }
}

// Write writes the specified number of lines to the file sequentially.
func (w *sequentialFileWriter) Write(totalLines int, filename string) error {
    // Create the output file
    file, err := w.fileWriter.CreateFile(filename)
    if err != nil {
        return err
    }
    defer w.fileWriter.FileClose(file)

    writer := w.fileWriter.NewBufferedWriter(file)
    for i := 0; i < totalLines; i++ {
        data := fmt.Sprintf("This is a line of data %d.\n", i)
        _, err := w.fileWriter.BufferedWriteString(writer, data)
        if err != nil {
            return err
        }
    }

    // Flush the buffer to ensure all data is written to the file.
    err = w.fileWriter.BufferedFlush(writer)
    if err != nil {
        return err
    }

    return nil
}

In this code sample, we've created a sequential file writer that writes lines of data sequentially to a file. Here's what's happening step by step:

We create a new instance of sequentialFileWriter using NewSequentialFileWriter.
The Write method takes the total number of lines to write and the filename.
We create the output file using w.fileWriter.CreateFile(filename) and defer its closure.
To optimize writing, we use a buffered writer created with w.fileWriter.NewBufferedWriter.
Inside a loop, we generate data for each line and write it using w.fileWriter.BufferedWriteString.
After writing all the lines, we ensure all data is flushed to the file with w.fileWriter.BufferedFlush.

This code demonstrates how to write data sequentially to a file while maintaining data integrity.

Parallel File Writer 🚀

What is Parallel Writing?
Parallel writing is like having multiple authors working on different parts of a book at the same time. It speeds up the process significantly.

When to Use it?

Utilize parallel writing when dealing with large datasets that need to be written quickly.
Ideal for scenarios where write speed is a top priority.
When your system can efficiently manage concurrent writes.

Pros:

Speed: Greatly improves write speed, especially for large datasets.
Efficiency: Efficiently utilizes system resources.
Scalability: Scales well with the number of cores or goroutines.

Cons:

A bit more complex to implement than sequential writing.
Data order might not be guaranteed.

Code Sample and Explanation:
filewriter/parallel/parallel.go

package parallel

import (
    "fmt"
    "os"
    "runtime"
    "sync"

    "github.com/frasnym/go-file-writer-example/filewriter"
)

type parallelFileWriter struct {
    fileWriter    filewriter.FileWriter
    maxGoRoutines int
}

func NewParallelFileWriter(fileWriter filewriter.FileWriter) filewriter.Writer {
    // Get the number of available CPU cores
    maxGoRoutines := runtime.GOMAXPROCS(0)

    return &parallelFileWriter{
        fileWriter:    fileWriter,
        maxGoRoutines: maxGoRoutines,
    }
}

func (w *parallelFileWriter) Write(totalLines int, filename string) error {
    // Create the output file
    file, err := w.fileWriter.CreateFile(filename)
    if err != nil {
        return err
    }
    defer w.fileWriter.FileClose(file)

    // Calculate the number of lines to be written by each worker
    linesPerTask := totalLines / w.maxGoRoutines

    var wg sync.WaitGroup
    errCh := make(chan error, w.maxGoRoutines)

    for i := 0; i < w.maxGoRoutines; i++ {
        wg.Add(1)
        go w.worker(i, file, &wg, linesPerTask, errCh)
    }

    // Close the error channel when all workers are done
    go func() {
        wg.Wait()
        close(errCh)
    }()

    // Collect and handle errors
    for err := range errCh {
        if err != nil {
            return err
        }
    }

    return nil
}

func (w *parallelFileWriter) worker(id int, file *os.File, wg *sync.WaitGroup, linesPerTask int, errCh chan error) {
    defer wg.Done()
    startLine := id * linesPerTask
    endLine := startLine + linesPerTask

    writer := w.fileWriter.NewBufferedWriter(file)
    for i := startLine; i < endLine; i++ {
        data := fmt.Sprintf("This is a line of data %d.\n", i)

        _, err := w.fileWriter.BufferedWriteString(writer, data)
        if err != nil {
            errCh <- err
            return
        }
    }

    w.fileWriter.BufferedFlush(writer)
}

In this code sample, we've created a parallel file writer that takes advantage of multiple goroutines to write lines of data concurrently to a file. Here's what's happening step by step:

We create a new instance of parallelFileWriter using NewParallelFileWriter. It determines the number of CPU cores available to decide how many goroutines to use.
The Write method takes the total number of lines to write and the filename.
We create the output file using w.fileWriter.CreateFile(filename) and defer its closure.
To optimize writing, we calculate the number of lines each worker should write based on the total number of lines and the available CPU cores.
We use a sync.WaitGroup to wait for all workers to complete their tasks and an error channel to collect and handle errors.
Inside a loop, we spawn multiple goroutines, each assigned to a worker function w.worker. These workers write their respective lines concurrently.
The worker function calculates the range of lines it's responsible for and writes data to the file using a buffered writer.
Once all workers are done, we close the error channel and collect and handle any errors.

This code demonstrates how to harness the power of parallelism to write data concurrently to a file, improving write speed significantly.

Parallel Chunk File Writer 🧩

What is Parallel Chunk Writing?
Think of it as writing chapters of a book in parallel while maintaining the order within each chapter.

When to Use it?

Choose parallel chunk writing for large datasets when both speed and data order are crucial.
Especially suitable when dealing with structured data that can be divided into chunks.
When your system can efficiently manage concurrent writes.

Pros:

Speed: Offers a significant speed boost for large datasets.
Data Order: Maintains data order within each chunk.
Scalability: Scales well with the number of cores or goroutines.

Cons:

A bit more complex to implement than sequential writing.

Code Sample and Explanation:
filewriter/parallelchunk/parallelchunk.go

package parallelchunk

import (
    "fmt"
    "runtime"
    "sync"

    "github.com/frasnym/go-file-writer-example/filewriter"
)

type parallelChunkFileWriter struct {
    fileWriter    filewriter.FileWriter
    maxGoRoutines int
}

func NewParallelChunkFileWriter(fileWriter filewriter.FileWriter) filewriter.Writer {
    // Get the number of available CPU cores
    maxGoRoutines := runtime.GOMAXPROCS(0)

    return &parallelChunkFileWriter{
        fileWriter:    fileWriter,
        maxGoRoutines: maxGoRoutines,
    }
}

func (w *parallelChunkFileWriter) Write(totalLines int, filename string) error {
    chunkSize := totalLines / w.maxGoRoutines
    var wg sync.WaitGroup

    for i := 0; i < w.maxGoRoutines; i++ {
        wg.Add(1)
        startLine := i * chunkSize
        endLine := startLine + chunkSize

        go w.writeChunkToFile(startLine, endLine, filename, &wg)
    }

    wg.Wait()

    return nil
}

func (w *parallelChunkFileWriter) writeChunkToFile(startLine, endLine int, filename string, wg *sync.WaitGroup) (err error) {
    file, err := w.fileWriter.CreateFile(fmt.Sprint(filename, "_", startLine))
    if err != nil {
        return err
    }
    defer w.fileWriter.FileClose(file)

    writer := w.fileWriter.NewBufferedWriter(file)

    for i := startLine; i < endLine; i++ {
        data := fmt.Sprintf("This is a line of data %d.\n", i)

        _, err = w.fileWriter.BufferedWriteString(writer, data)
        if err != nil {
            return
        }
    }

    w.fileWriter.BufferedFlush(writer)

    wg.Done()

    return
}

In this code sample, we've created a parallel chunk file writer that divides data into smaller chunks and writes them concurrently to a file while maintaining data order within each chunk. Here's what's happening step by step:

We create a new instance of parallelChunkFileWriter using NewParallelChunkFileWriter. It determines the number of CPU cores available to decide how many goroutines to use.
The Write method takes the total number of lines to write and the filename.
To optimize writing, we calculate the number of lines each worker should write based on the total number of lines and the available CPU cores.
We use a sync.WaitGroup to wait for all workers to complete their tasks.
Inside a loop, we spawn multiple goroutines, each assigned to a worker function w.writeChunkToFile. These workers write their respective chunks of lines concurrently while maintaining data order within each chunk.
The worker create the output file using w.fileWriter.CreateFile(filename) and defer its closure.
The worker function calculates the range of lines it's responsible for and writes data to the file using a buffered writer.
Once all workers are done, we wait for the sync.WaitGroup to signal that they've completed their

tasks.

This code demonstrates how to achieve both speed and data order within chunks by dividing data and utilizing parallelism effectively.

Conclusion 🤝

As a beginner developer, choosing the right method for writing large data into files depends on your specific use case:

Sequential Writing is suitable for simplicity and data integrity but is slower for large datasets.
Parallel Writing is ideal when write speed is crucial, and you can manage concurrent writes efficiently, even if data order isn't a top priority.
Parallel Chunk Writing is the sweet spot, offering both speed and data order within chunks for large datasets, but it comes with a moderate implementation complexity.

Consider your project's requirements, system capabilities, and the balance between speed and data integrity when making your choice. Happy coding! 🚀📦

Want to connect?

Got questions or just want to chat? Connect with me on Twitter/X anytime. Can't wait to chat with you! 💬

Secure File Transfer Made Easy: Connect to SFTP Servers with Golang

Frastyawan Nym — Mon, 28 Aug 2023 08:07:21 +0000

Intro
WHAT and WHY is SFTP?
How to connect to an SFTP
Something you can do after
Conclusions
Footnote

Intro

In this article, we'll show you how to connect to an SFTP server using Golang. SFTP is a secure file transfer protocol that allows you to transfer files over the internet securely 🔒. Golang is a programming language that's easy to learn and use, making it an excellent choice for building applications that require secure file transfer 💻.

We'll start by explaining what SFTP is and why it's important 📡. Next, we'll walk you through the steps to connect to an SFTP server using Golang, including establishing an SSH connection, creating an SFTP client, and performing file transfer operations 🚀.

By the end of this article, you'll have a good understanding of how to connect to an SFTP server using Golang and be able to apply this knowledge to your own projects 📝.

WHAT and WHY is SFTP?

File transfer is an essential aspect of modern digital communication, allowing individuals and organizations to share data across networks and devices. However, with the increasing threat of cyberattacks and data breaches, secure file transfer has become more critical than ever 🔒.

This is where Secure File Transfer Protocol (SFTP) comes in, providing a secure and reliable way to transfer files over the internet 🌐.

How to connect to an SFTP

First, you'll need to import the "github.com/pkg/sftp" package in your Go code. This package provides an SFTP client implementation.
Next, you'll need to establish an SSH connection to the SFTP server. You can use the "golang.org/x/crypto/ssh" package to do this. Here's an example code snippet:

config := &ssh.ClientConfig{
    User: "username",
    Auth: []ssh.AuthMethod{
        ssh.Password("password"),
    },
    HostKeyCallback: ssh.InsecureIgnoreHostKey(),
}

conn, err := ssh.Dial("tcp", "sftp.example.com:22", config)
if err != nil {
    log.Fatal("Failed to dial: ", err)
}

client, err := sftp.NewClient(conn)
if err != nil {
    log.Fatal("Failed to create SFTP client: ", err)
}
defer client.Close()

In this example, we're using a password-based authentication method. You can also use other authentication methods, such as public key authentication 🔑.

Something you can do after

Once you've established the SSH connection and created an SFTP client, you can use the client to perform SFTP operations, such as uploading and downloading files. Here's an example code snippet that uploads a file to the SFTP server:

localFile, err := os.Open("/path/to/local/file")
if err != nil {
    log.Fatal("Failed to open local file: ", err)
}
defer localFile.Close()

remoteFile, err := client.Create("/path/to/remote/file")
if err != nil {
    log.Fatal("Failed to create remote file: ", err)
}
defer remoteFile.Close()

_, err = io.Copy(remoteFile, localFile)
if err != nil {
    log.Fatal("Failed to upload file: ", err)
}

In this example, we're opening a local file, creating a remote file on the SFTP server, and then copying the contents of the local file to the remote file.

Conclusions

In this article, we've shown you how to connect to an SFTP server using Golang. By following the steps outlined in this article, you now have the knowledge and skills to automate file transfers, improve efficiency, and enhance security in your data exchange processes 🚀.

We encourage you to apply what you've learned and explore further on your own. Golang has a vibrant community and a wealth of resources available, so don't hesitate to reach out and learn more 💻.

Thank you for reading, and we hope this article has been informative and useful to you 📝.

Footnote

Full code is available on github 🌐

Want to connect?

Twitter/X

Building a Deep Nested CLI Application with Cobra in Golang

Frastyawan Nym — Wed, 02 Aug 2023 08:30:01 +0000

Continuing my previous article about Cobra , the powerful Golang library for building command-line interfaces, we're diving deeper into the world of CLI development.

In this installment, we'll explore how to take advantage of Cobra's capabilities to create a deeply nested CLI application. By organizing commands and subcommands in a hierarchical structure, we can make our CLI more user-friendly, intuitive, and flexible for handling complex tasks. Join me as we embark on this journey of building a feature-rich CLI tool that showcases the true power of Cobra in Golang. Let's get started!

Implementing Deep Nesting

To achieve deep nesting, we'll add subcommands under the existing config command. Let's create subcommands under subcommand: deepsubcommand.

// cmd/deepsubcommand.go
package cmd

import (
    "fmt"

    "github.com/spf13/cobra"
)

var deepSubCmd = &cobra.Command{
    Use:   "deepsubcommand",
    Short: "A brief description of the deepsubcommand",
    Long: `A longer description that explains the deepsubcommand
in detail. For example, you can mention how it complements
the main command and what options it supports.`,
    Run: func(cmd *cobra.Command, args []string) {
        // This function will be executed when the "deepsubcommand" is called
        fmt.Println("Running the deepsubcommand!")
    },
}

func init() {
    subCmd.AddCommand(deepSubCmd)
}

Lest try it with:

$ go run main.go subcommand deepsubcommand
> Running the deepsubcommand!

Conclusion:

In this blog post, we explored how to create a deep nested CLI application using the Cobra library in Golang. By organizing commands and subcommands hierarchically, we can build user-friendly CLIs that enhance the overall user experience. Cobra's intuitive API and support for commands, subcommands, and flags make it a powerful choice for building CLI applications in Golang.

In your own projects, you can extend this deep nested CLI application further and customize it to suit your specific use cases. Have fun experimenting with different configurations and commands to create a versatile and user-friendly CLI tool!

Footnote:

Full code is available on github

Getting Started with Cobra: Creating Multi-Level Command Line Interfaces in Golang

Frastyawan Nym — Mon, 31 Jul 2023 10:29:40 +0000

Introduction:

Creating a CLI using Cobra in Golang is a great choice, and nesting commands can help organize and structure your CLI effectively. Below, I'll guide you through the process of creating a nested CLI using Cobra.

What is Cobra?

Cobra is a robust and flexible Golang library for building CLI applications. It provides an elegant and easy-to-use interface to define commands, flags, and arguments, making it a popular choice among Golang developers for creating CLI applications.

Prerequisites:

Before we begin, ensure you have Go (Golang) installed and have set up your workspace properly. You can install Cobra by executing the following command:

go get -u github.com/spf13/cobra@latest

Getting Started:

Let's start by creating a new Golang project in your desired workspace. Once your project is set up, we'll initialize a Cobra application by running the following command:

cobra init [your-cli-app-name]

This will create a basic structure for your CLI application, including the necessary files and folders.

Defining the Root Command:

The root command is the entry point of your CLI application. It is the command executed when users run your application without any subcommands. Open the cmd/root.go file, and let's define the root command using Cobra:

// cmd/root.go
package cmd

import (
    "fmt"
    "github.com/spf13/cobra"
)

var rootCmd = &cobra.Command{
    Use:   "[your-cli-app-name]",
    Short: "A brief description of your CLI application",
    Long: `A longer description that explains your CLI application
in detail. For example, you can mention the various commands
available and what they do.`,
    Run: func(cmd *cobra.Command, args []string) {
        // This function will be executed when the root command is called
        fmt.Println("Welcome to [your-cli-app-name]! Use --help for usage.")
    },
}

func Execute() {
    if err := rootCmd.Execute(); err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
}

You can try to run it:

$ go run main.go                                                          
> Welcome to [your-cli-app-name]! Use --help for usage.

Adding a Nested Command:

Next, let's add a nested command to our CLI application. Nested commands are subcommands that provide additional functionality under the main command. For example, let's create a subcommand called "subcommand" for our application. Open the cmd/subcommand.go file and define the nested command:

// cmd/subcommand.go
package cmd

import (
    "fmt"
    "github.com/spf13/cobra"
)

var subCmd = &cobra.Command{
    Use:   "subcommand",
    Short: "A brief description of the subcommand",
    Long: `A longer description that explains the subcommand
in detail. For example, you can mention how it complements
the main command and what options it supports.`,
    Run: func(cmd *cobra.Command, args []string) {
        // This function will be executed when the "subcommand" is called
        fmt.Println("Running the subcommand!")
    },
}

func init() {
    rootCmd.AddCommand(subCmd)
}

Lets try it again:

$ go run main.go subcommand           
> Running the subcommand!

Conclusion:

Congratulations! 🎉 You have successfully created a CLI application using Cobra with a nested command. You can continue to expand your application by adding more subcommands, flags, and arguments to meet your specific needs.

In this blog post, we've only scratched the surface of what you can achieve with Cobra. It's a powerful library that allows you to create professional-grade CLIs with ease. Experiment with various options, customize your commands, and explore more features provided by Cobra.

Happy coding! 👨‍💻

Footnote:

Full code is available on github 🤝

Access your Golang Private Repository

Frastyawan Nym — Thu, 20 Jul 2023 05:17:02 +0000

Add your ssh to you git host
Update your GOPRIVATE
go env -w GOPRIVATE="github.com/<OrgNameHere>/*"
Update your gitconfig. This example for bitbucket:
git config --global url."git@bitbucket.org:".insteadOf "https://bitbucket.org/"

Add your ssh key with ssh-agent.You have to apply

eval "$(ssh-agent -s)"
> Agent pid 5867
ssh-add <private-ssh-key-file-name> // eg. id_rsa (not .pub)
> Enter passphrase for /home/you/.ssh/id_rsa: [] Identity added: /home/you/.ssh/id_rsa (/home/you/.ssh/id_rsa)

Try run go mod tidy on your project

Database Indexes, please be careful when using it!

Frastyawan Nym — Wed, 06 Apr 2022 05:28:52 +0000

Lets do disclaimer before we start 😉.

This is my current understanding of Database Index.
If there is any mistake that write that lead to misunderstanding, please let me know and I will update this post ASAP.

Let's begin 🚀

What is database index?

Database index is a built-in tools that will help us for searching data inside database. Index will quickly locate record of our queries.

The problem

To help to understand, I will make up some use case. Example, we have a users table that have id, full_name & gender

That users table will have 1000 rows. 2 of them is male and the rest (998) of them is female.
Say that we need to find our male user, we will create query something like this SELECT * FROM users WHERE gender = 'male'.
After we executed that query, we will have result of 2 users.

Behind the scene, database engine will search all of 1000 rows to find that 2 rows.
Yes, that sound inefficient. But that is how database engine works. 😵

What is the solution?

The solution is to use Database Index. 💡
With index, database engine will make some of algorithm that map our table based on the index we chose.

Okay, so we create an index on gender column inside users table.
Back again to our query, now if we execute SELECT * FROM users WHERE gender = 'male', we will get 2 users again as result (of course!).

The difference is, right now database engine will search on 2 rows to find that 2 rows.
Now that sound efficient❗

But wait...

After we know that, maybe there is a thinking like this:

"let's create index for all of our column, so we can get quick result every time we search something 🤔"

Thats right, you will get a quick query every search.
But please don't over-used it. Even though Index have upside, Index also have downside.

If we create an index, behind the scene database engine will write additional data to our database.

Lets back again to our users table. The calculation is like this:

2 index (1 for ID, 1 for gender) * 1000 rows = 2000
# PK is always get index

Imagine if we put index to all of our column

4 index (ID, gender, full_name, created_at) * 1000 rows = 4000
# Thats only 4 columns.
# The more column we create, the more usage it will take when CREATE, UPDATE, DELETE, etc.

the conclusion is: please use it wisely. 😇

Bonus: How to create index

PostgreSql

CREATE TABLE "customers" (
  "id" SERIAL PRIMARY KEY,
  "gender" varchar,
  "full_name" varchar,
  "created_at" timestamp
);

CREATE INDEX ON "customers" ("gender");

MySql

CREATE TABLE `customers` (
  `id` int PRIMARY KEY AUTO_INCREMENT,
  `gender` varchar(255),
  `full_name` varchar(255),
  `created_at` timestamp
);

CREATE INDEX `customers_index_0` ON `customers` (`gender`);

TypeScript Linting and Code Formatter

Frastyawan Nym — Sat, 10 Apr 2021 03:23:12 +0000

Btw, before I forgot to mention it.
This is a series post about Create Express API TypeScript Boilerplate and this is the first post.
After the second, third, etc. releases, I will mention it here. 👍

Still learning it thought, how to create a series post on this site. 😂

Let's Get Started! 🏃

TL;DR 🤝: Source Code

On this project, we will use the famous Visual Studio Code.
So please install it first so you can expect same result in the end 😁.

And to avoid unexpected things happened, let's install some extension to VSCode.

ESLint

Prettier

Setup Lint 🚨

Install ESLint package: npm i -D eslint
- This is the main package. This package support not only JavaScript but TypeScript as well.
Install TypeScript package: npm i -D typescript
- This is a package that we will use to work with TypeScript.
Initialize TypeScript: npx tsc --init
- We can initialize typescript with this command. This command will be make a file that called tsconfig.json.
- Inside that file, we can specify what config we need. Let's just leave it by default for now. Example:
Initialize ESLint: npx eslint --init
- Now initialize the ESLint configuration. You can copy my settings below for TypeScript support. Example
- Try it Out! 🚀
- How to make ESLint catch error on code that we write: npx eslint src/*.ts. Example
- How to fix the error: npx eslint src/*.ts --fix. Example:

Setup Code Formatter 🎨

We will use Prettier to help us here
Install the required package: npm i -D prettier-eslint
Next we will install cli version of prettier-eslint: npm i -D prettier-eslint-cli
- Try it Out! 🚀
- How to format the file: npx prettier-eslint 'src/**/*.ts'. Example:
- How to format and actually change the file: npx prettier-eslint 'src/**/*.ts' --write. Example:
Add additional settings on Prettier to follow ESLint
Create .prettierrc.yaml file. We choose YAML format so we can add comments. There is others format if you want to set differently: Link
Set the settings' value on .prettierrc.yaml:

semi: false # We don't need semicolons at the end of line singleQuote: true # Use single quote
Add additional settings too on .vscode/settings.json: "editor.formatOnSave": true
- If you don't have this file, just create one.
- This setting will tell VSCode to format the code on file save.

Closing

That's all I can share with you for now 😊
If you have any question, you can post it here.
Or maybe you can reach me on my Twitter 👋

Until then... 🚀

Connecting NextJS Mini Project with Google Spreadsheets as Database

Frastyawan Nym — Thu, 11 Feb 2021 04:12:53 +0000

Did you know that we can use Google Sheets as our database?
Yes! We can do that 👍.

Today, I will share step by step to do it.
I will use one of the Javascript Frontend Library to do this, it's NextJS.

So, let's get started! 🚀

Create Your Google Project

Go to your google console page
Create your project, or you can visit through this Link 🔗
🔙 Back to APIs & Services page, click on ENABLE APIS AND SERVICES
Search for Google Sheets API then enable it 🖱️
🔙 Back again to APIs & Services page, click Credentials, CREATE CREDENTIALS, Service Account. Or through this Link 🔗
Input required field and you can ignore optional field (of course 😅)
🔙 Back to your Credentials page, under Service Accounts you will see your email. Copy that!
Open your Service Account. Click on ADD KEY, choose JSON as type. Then your credentials will be automatically downloaded. This will be used when connecting you NextJS to the sheets.
Now you can create new spreadsheets or use your existing one, Open it
Share it to your Service Account Email you just copied a second ago
This step is done 🎊

Create NextJS App

Reference: NextJS Docs

Create an app```

npx create-next-app [APP_NAME]

or

yarn create next-app [APP_NAME]

2. 🏃‍♂️ Run your app with `npm run dev`, wait until server started
3. Without making any changes, you can access you page on **localhost:3000**. You will see awesome Next starter page 😎
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/1s90tzp88i3tgxq1f800.JPG)
4. Continue 🚀, Setup Environment Variables
5. Create **env.local** file. You will need set this key-value pair. The value you can get on downloaded .JSON file before from google console```


GOOGLE_SHEETS_PRIVATE_KEY=[YOUR KEY]
GOOGLE_SHEETS_CLIENT_EMAIL=[YOUR ACCOUNT EMAIL]
SPREADSHEET_ID=[YOU CAN GET THIS ON URL OF YOUR SHEETS]

We will need googleapis package to help us doing this, so install it```

npm i googleapis

7. Next, create a file for connecting to our sheet. For me is under **libs/sheets.js**

import { google } from 'googleapis';
export async function getEmojiList() {
try {
const target = ['https://www.googleapis.com/auth/spreadsheets.readonly'];
const jwt = new google.auth.JWT(
process.env.GOOGLE_SHEETS_CLIENT_EMAIL,
null,
(process.env.GOOGLE_SHEETS_PRIVATE_KEY || '').replace(/\n/g, '\n'),
target
);

const sheets = google.sheets({ version: 'v4', auth: jwt });
const response = await sheets.spreadsheets.values.get({
  spreadsheetId: process.env.SPREADSHEET_ID,
  range: 'emoji', // sheet name
});

const rows = response.data.values;
if (rows.length) {
  return rows.map((row) => ({
    title: row[2],
    subtitle: row[3],
    code: row[4],
    browser: row[5],
    short_name: row[17],
    emojipedia_slug: row[18],
    descriptions: row[19],
  }));
}

} catch (err) {
console.log(err);
}
return [];
}

### Description:
- We will specify our target connection

const target = ["https://www.googleapis.com/auth/spreadsheets.readonly"];

- Declare our jwt for authentication

const jwt = new google.auth.JWT(
process.env.GOOGLE_SHEETS_CLIENT_EMAIL,
null,
(process.env.GOOGLE_SHEETS_PRIVATE_KEY || '').replace(/\n/g, '\n'),
target
);

- Get the sheet data, don't forget to change **sheet name**

const sheets = google.sheets({ version: 'v4', auth: jwt });
const response = await sheets.spreadsheets.values.get({
spreadsheetId: process.env.SPREADSHEET_ID,
range: 'emoji', // sheet name
});

- Declare the row as your sheet data

const rows = response.data.values;
if (rows.length) {
return rows.map((row) => ({
title: row[2],
subtitle: row[3],
code: row[4],
browser: row[5],
short_name: row[17],
emojipedia_slug: row[18],
descriptions: row[19],
}));
}


That was a lot of text, now let's get the easy part 😬
# Populate Your Data
This is my `index.js` file

import Head from 'next/head';
import { getEmojiList } from '../libs/sheets';

export default function IndexPage({ emojis }) {
return (
<>

Title - FrasNym

{emojis[0].title}
</>
);
}

export async function getStaticProps(context) {
const emojis = await getEmojiList();
return {
props: {
emojis: emojis.slice(1, emojis.length), // remove sheet header
},
revalidate: 1, // In seconds
};
}

You can already see our data from `emojis` variable that passed through `props`.

As you expect, We load the data from **Google Sheets** via our `getEmojiList` function. Then we call that function from `getStaticProps` function and pass it as props to our component.

On `getStaticProps`, we pass `revalidate: 1` so we hope that NextJS page will render when our sheets updated (At most once every second).

# Summary
This tools will help us to do a small project with database easily.
💡 FYI, You can insert to sheet via API too. You can connect this sheet with Your NodeJS with [google-spreadsheet](https://www.npmjs.com/package/google-spreadsheet) package, for example.

# Closing
That's all I can share with you for now 😊
If you have any question, you can post it here.
Or maybe you can reach me on my [Twitter](https://twitter.com/frasnym) 😁

Until then... 🚀

How to Create Dockerized NodeJS with MySQL Database

Frastyawan Nym — Tue, 09 Feb 2021 09:48:58 +0000

TLDR;
You can access the code here on Github
(step by step how to run included 🤝)

Hello 👋, This will be my first post on this forum.
I will show you guys how to create a NodeJS-MySQL database with help of Docker.

FYI, I just learn using docker too and I found that docker is very good to help development process faster ⚡.

Here we go

I will assume that you already know what docker is and already installed it on your machine. I will use docker-compose to create this project.

Create MySQL Container, I'm using MySQL image to help me build this container.
Create docker-compose.yaml file. I this file we will be specifying our docker app.

version: '3.8'

services: 
  mysqldb:
    image: mysql
    restart: always
    env_file: ./.env
    environment:
      MYSQL_ROOT_PASSWORD: $MYSQL_ROOT_PASSWORD
      MYSQL_DATABASE: $MYSQL_DATABASE
    ports:
      - $MYSQL_LOCAL_PORT:$MYSQL_DOCKER_PORT
    volumes: 
      - db-config:/etc/mysql
      - db-data:/var/lib/mysql
      - ./db/backup/files/:/data_backup/data

volumes: 
  db-config:
  db-data:

Description:

version: '3.8': This first line of code must be provided on docker-compose.yaml file. This will tell docker which version of docker we used
services:: Inside this tag we will tell docker what service we will make
mysqldb:: The first service is mysqldb. You're free to choose the name
image: mysql: We tell docker what image we will use. mysql is official image for MySQL
restart: always: This command will tell docker always restart the container regardless of the exit status
env_file: ./.env: We specify our .env path
environment: ON this tag we provide our MySQL connection setting
MYSQL_ROOT_PASSWORD:: Specify password for root username
$MYSQL_ROOT_PASSWORD is key from .env
MYSQL_DATABASE: $MYSQL_DATABASE: create initial database
ports:: we specify what port docker will be used. We will specify 2 port
$MYSQL_LOCAL_PORT: The first one is what port on our machine will be used
$MYSQL_DOCKER_PORT: The second one is what port we used inside docker container
volumes:: Volume will help us to keeping our data alive though restart
db-config:/etc/mysql: Volume to save config
db-data:/var/lib/mysql: Volume to save database data
./db/backup/files/:/data_backup/data: Mount bind backup data

That a lot of typing already LOL 😂

Let's continue 🚀

Now we make our NodeJS app, inside of app folder
You can initialize npm as usual
Here we will be create a Dockerfile, this will help us to create image for NodeJS App

FROM node:14

WORKDIR /app

COPY /app/package.json .

RUN npm install

COPY /app .

EXPOSE 3000

CMD [ "npm", "start" ]

Description:

FROM node:14: Specify base image for node. I take official image of NodeJS
WORKDIR /app: Define working directory of docker. Our app will be placed on this folder inside docker
COPY /app/package.json .: Copy our package.json file to our working directory (declared by dot(.) code)
RUN npm install: Install npm dependency as always
COPY /app .: Next we will copy our rest of file to working directory
EXPOSE 3000: We expose port to access via localhost
CMD [ "npm", "start" ]: Specify script to run after image is builded

Did you confused? If yes, me too when first learning this docker thing 😁

Continue 🚀

Back to our docker-compose.yaml file

Now we will define our NodeJS App on docker-compose

  app:
    build:
      context: .
      dockerfile: ./app/Dockerfile
    image: node-mysql-app
    depends_on:
      - mysqldb
    stdin_open: true
    tty: true

volumes: 
  db-config:
  db-data:

The command is more or less the same

Description

app:: This is our second service's name
build:: For custom image we will use build
context: .: Specify the file PATH
dockerfile: ./app/Dockerfile: This will be our previous Dockerfile
image: node-mysql-app: Define name to our custom image
depends_on: - mysqldb: This will tell docker that the second service will depends on first service
stdin_open: true: This will tell docker that keep open the terminal after complete building container
tty: true: Same as above
volumes:: Last, we tell docker that we have named volume inside our service
db-config:: This is the name of the named volume
db-data:: This is the name of the named volume

And there we go, that's all is needed for creating our docker app 👍
Lastly, we will execute this command to start our apps: docker-compose up
After success, you can access your app on localhost:3000 🐳

Summary

With this, We don't need to install MySQL locally to start development right. I think this is the power of docker "To speed up our development process"
That's all I can share with you for now 😊
If you have any question, you can post it here. I hope that I can help to answer it 💪

Or maybe you can reach me on my Twitter
I love to build a new connection 😁

Until then... 🚀