DEV Community: Laba Subagia

Shortcut Alternative to Fn for Linux

Laba Subagia — Sun, 14 Apr 2024 01:45:32 +0000

My laptop keyboard is not working right now, and I use my backup external keyboard that doesn't have Fn Key. I use several alternative shortcuts. You can customize all the Keys based on your preference.

Prerequisite

amixer from alsa-utils
brightnessctl

Volume Up
Key: Ctrl + Up (keypad)
Command: amixer -q -D pulse sset Master 5%+

Volume Down
Key: Ctrl + Down (keypad)
Command: amixer -q -D pulse sset Master 5%-

Brightness Up
Key: Ctrl + Right (keypad)
Command: brightnessctl set +5%

Brightness Down
Key: Ctrl + Left (keypad)
Command: brightnessctl set 5%-

These shortcuts were tested on LinuxMint Xfce. I will update this article when I make a new shortcut. I hope this is useful for you.

Thank you.

Tools for Linux Distro Hoppers

Laba Subagia — Wed, 27 Mar 2024 14:56:11 +0000

Linux is one of the most popular operating systems (technically, Linux is just the Kernel) and has many types (distribution/distro). It may be unclear to choose which one to use, from beginner-friendly distro to those used for special needs.
Linux users who use Linux as a daily driver usually have tried a lot of distros before sticking to one.

Distro Hoppers

Distro Hoppers is a term for users who often change distros regularly. The reason can vary: try new challenges, different features and different purposes of the distro. Distro hopping is sometimes considered bad because it requires much time and offers little benefit.

This article will discuss tools and advice for Distro Hoppers.

Virtual Machine: Virtualbox / KVM

The virtual Machine application is useful for trying the Linux distro before you even consider installing it or burning it on your flash disk. It is important to have a taste before installing it so you're not wasting so much time if you don't like the distro in the first place.

Backups

Backup is important to prevent data loss after installing a new Linux distribution. There are several ways.

Using backup tools, e.g. rsync, Timeshift, etc.
Separate partition for data, or separate partition for /home directory. With this approach, you must mount that partition after installing the new distribution.
Save all data to the cloud. Several providers are out there, e.g., pCloud and MEGA. Just remember only to use a provider that has a Linux Client app.

I recommend the last approach because saving important data on a local computer is not good practice.

USB Bootable: Ventoy

Ventoy is an open-source tool to create a bootable USB drive for ISO/WIM/IMG/VHD(x)/EFI files. Using Ventoy, we can create live USB for multiple distros in one flash disk. This is a revolutionary feature compared to other tools, e.g., Etcher, Rufus, etc, that can only create one live USB at a time.
This tool is so important. Don't call yourself a distro hopper if you don't know this tool.

Package Manager: Flatpak

Hopping from one distro to another with a different package manager might require some time to adapt. Using a package manager that can be installed on most distro is one way to help you get to work faster. Flatpak is one of them; other alternative are Snap, Nix or Homebrew. Flatpak is a good starter, and if you have a bunch of free time, I suggest trying Nix.

Development: Docker with DevContainer

If you are using Linux, I think it's safe to assume that your job is somewhat related to programming. Setting up a development environment for a Software Engineer can be a hurdle. The development environment might require installing many tools, e.g., programming language, database, monitoring tools, etc.
Docker with DevContainer is one of the fastest solutions to create a development environment faster, even after hopping from several distros.

Docker is a containerization tool that ships your app with all the necessary dependencies. Using DevContainer makes developing software inside a container possible. Several editors, e.g., Visual Studio Code and JetBrains, already support DevContainer. If you use other editors like VIM or Helix, you might develop software directly inside the container or need to find another workaround. But if you are too lazy to configure VIM like me, use VSCode.

Honorable Mentions

Distrobox

Distrobox is a tool that enables us to try Linux distro CLI, including their package manager. This requires a containerization tool (e.g., Docker). In Windows, this can be achieved using WSL (Windows Subsystem for Linux)

EasyBCD

EasyBCD is a Windows app that can modify the Bootloader if you dual-boot with Windows. I use this tool to remove Linux Grub when distro hopping.

Conclusion

Distro Hopping is not always bad, especially if you want to learn something new. Do this responsibly so it doesn't bother your work or your time with family. Utilize the tools above to minimize the time wasted, or even better if there is a separate computer just for that.

OK, that's all for now. I hope you gain some insight and become better Distro Hoppers.

Thank you.

Simple utility function that will make you handle pointers in Go easier

Laba Subagia — Mon, 12 Feb 2024 14:57:15 +0000

Using Pointer in Go is sometimes inevitable. One case is when I started to learn how to use Go. At that time, the team was migrating the legacy Node.js Backend application to Go. The frontend application is using Javascript.

One nature that was different at that time was dynamic typing language vs static typing language. One thing that is quite annoying is the ability of dynamic-type language to use partial objects to update data.

//let's say all fields in the database table are like this
const user = {
  id: 1,
  name: 'John',
  address: 'Indonesia',
  phone: '+62 881 112 312 xxx',
  isBanned: true
}


// then from frontend updated like this
const updateInput = {
  name: 'Erick'
}

// update
const updatedUser = { ...user, ...updateInput }

console.log(updatedUser)

// Output:
// {
//   address: "Indonesia",
//   id: 1,
//   isBanned: true,
//   name: "Erick",
//   phone: "+62 881 112 312 xxx"
// }

This works fine when the data communication happens between two dynamic-typed languages. But the introduction of Go to the Playground brings a headache. The decision at that time was we were not allowed to change the front end because if it still worked fine, don't touch it.

So, we need to take the burden to the backend. It's challenging to do because Go uses default values for its data types.

package main

import "fmt"

type User struct {
    ID       int
    Name     string
    Phone    string
    Address  string
    IsBanned bool
}

func (u *User) Update(input User) {
    u.Name = input.Name
    u.Phone = input.Phone
    u.Address = input.Address
    u.IsBanned = input.IsBanned
}

func main() {

    // Imagine this is current user data from the database
    user := User{
        ID:       1,
        Name:     "John",
        Phone:    "+62 881 112 312 xxx",
        Address:  "Indonesia",
        IsBanned: true,
    }

    //This is the update payload
    payload := User{
        Name: "Eric",
    }

    user.Update(payload)

    fmt.Printf("%+v\n", user)
}


// Output: {ID:1 Name:Eric Phone: Address: IsBanned:false}

This becomes a headache for people who come from dynamic typing language, and things don't work out. The result in the Go source code is that Go treats the default value as the same as the regular value so that it will replace the previous value.

To resolve this issue, we need something that can mimic the behavior of an object in dynamic typing language. This is when a Pointer comes in. A pointer can indicate the absence of a field in a struct and avoid using a default value (maybe not entirely).

package main

import "fmt"

type User struct {
    ID       *int   
    Name     *string 
    Phone    *string 
    Address  *string 
    IsBanned *bool   
}

func (u *User) Update(input User) {
    if input.Name != nil {
        u.Name = input.Name
    }
    if input.Phone != nil {
        u.Phone = input.Phone
    }
    if input.Address != nil {
        u.Address = input.Address
    }
    if input.IsBanned != nil {
        u.IsBanned = input.IsBanned
    }
}

func main() {

    // Imagine this is current user data from the database
    ID := 1
    name := "John"
    phone := "+62 881 112 312 xxx"
    address := "Indonesia"
    isBanned := true

    user := User{
        ID:       &ID,
        Name:     &name,
        Phone:    &phone,
        Address:  &address,
        IsBanned: &isBanned,
    }

    //This is the update payload
    updatedName := "Eric"
    payload := User{
        Name: &updatedName,
    }

    user.Update(payload)

    fmt.Printf("%+v\n", user)
}

// Output: {ID:0xc0000aa010 Name:0xc000096070 Phone:0xc000096050 Address:0xc000096060 IsBanned:0xc0000aa018}

That is the code that is updated using Pointer. But what the hell is going on with the output? That happened because the pointer holds a memory address, and the fmt.Printf cannot handle that directly. So there is extra work that needs to be done.

package main

import (
    "encoding/json"
    "fmt"
)

type User struct {
    ID       *int
    Name     *string
    Phone    *string
    Address  *string
    IsBanned *bool
}

func (u User) String() string {
    bytes, _ := json.Marshal(u)
    return string(bytes)
}

func (u *User) Update(input User) {
    if input.Name != nil {
        u.Name = input.Name
    }
    if input.Phone != nil {
        u.Phone = input.Phone
    }
    if input.Address != nil {
        u.Address = input.Address
    }
    if input.IsBanned != nil {
        u.IsBanned = input.IsBanned
    }
}

func main() {

    // Imagine this is current user data from the database
    ID := 1
    name := "John"
    phone := "+62 881 112 312 xxx"
    address := "Indonesia"
    isBanned := true

    user := User{
        ID:       &ID,
        Name:     &name,
        Phone:    &phone,
        Address:  &address,
        IsBanned: &isBanned,
    }

    //This is the update payload
    updatedName := "Eric"
    payload := User{
        Name: &updatedName,
    }

    user.Update(payload)

    fmt.Printf("%+v\n", user)
}

// Output: {"ID":1,"Name":"Eric","Phone":"+62 881 112 312 xxx","Address":"Indonesia","IsBanned":true}

Finally, everything that is required for the Backend migration is met.

But, there is something still bogging me. Using a pointer is considered harmful because improper use can lead to nil dereference error (in other programming languages, it might be known as Null Pointer Exception). This error can stop the entire program.

We need to carefully check the value before using it.

if input.Name != nil {
    u.Name = input.Name
}
if input.Phone != nil {
    u.Phone = input.Phone
}
if input.Address != nil {
    u.Address = input.Address
}
if input.IsBanned != nil {
    u.IsBanned = input.IsBanned
}

Assigning value is cumbersome because we need to declare a variable before we can use the value.

ID := 1
name := "John"
phone := "+62 881 112 312 xxx"
address := "Indonesia"
isBanned := true

user := User{
    ID:       &ID,
    Name:     &name,
    Phone:    &phone,
    Address:  &address,
    IsBanned: &isBanned,
}

We need a way to resolve this issue.

Using a simple utility function to make it safer and easier

// get the memory address of a value
func Ptr[T any](value T) *T {
    return &value
}

// get the value of the pointer, or else use another value
func ValOrElse[T any](ptr *T, defaultVal T) T {
    if ptr == nil {
        return defaultVal
    }
    return *ptr
}

// set pointer value safely
func SetPtr[T any](ptr **T, newPtr *T) {
    if ptr == nil || newPtr == nil {
        return
    }
    if *ptr == nil {
        *ptr = new(T)
    }
    **ptr = *newPtr
}

By using the functions above, we can use pointers safer or at least easier than the previous.

package main

import (
    "encoding/json"
    "fmt"
)

type User struct {
    ID       *int
    Name     *string
    Phone    *string
    Address  *string
    IsBanned *bool
    IsActive *bool
}

func (u User) String() string {
    bytes, _ := json.Marshal(u)
    return string(bytes)
}

func (u *User) Update(input User) {
    SetPtr(&u.Name, input.Name)
    SetPtr(&u.Phone, input.Phone)
    SetPtr(&u.Address, input.Address)
    SetPtr(&u.IsBanned, input.IsBanned)
}

func main() {

    // Imagine this is current user data from the database
    user := User{
        ID:       Ptr(1),
        Name:     Ptr("Name"),
        Phone:    Ptr("+62 881 112 312 xxx"),
        Address:  Ptr("Indonesia"),
        IsBanned: Ptr(true),
    }

    //This is the update payload
    payload := User{
        Name: Ptr("Eric"),
    }

    user.Update(payload)

    fmt.Printf("%+v\n", user)
    fmt.Println("is user active?", ValOrElse(user.IsActive, false))
}

// get the memory address of a value
func Ptr[T any](value T) *T {
    return &value
}

// get the value of the pointer, or else use another value
func ValOrElse[T any](ptr *T, defaultVal T) T {
    if ptr == nil {
        return defaultVal
    }
    return *ptr
}

// set pointer value safely
func SetPtr[T any](ptr **T, newPtr *T) {
    if ptr == nil || newPtr == nil {
        return
    }
    if *ptr == nil {
        *ptr = new(T)
    }
    **ptr = *newPtr
}

// Output:
// {"ID":1,"Name":"Eric","Phone":"+62 881 112 312 xxx","Address":"Indonesia","IsBanned":true,"IsActive":null}
// is user active? false

Using some of the utility above might be subjective, but you can use it if you think the same. For me, it helps me to worry less about nil pointer issues.

Using a pointer above might not be the best way to handle my case. I think a better way is to use an approach using a type like sql.NullString. But that will be a topic for the next discussion.

That's all from me. Stay tuned.

Thank you.

Pointer and Nil in Go: Reasons why we should be wary

Laba Subagia — Sun, 11 Feb 2024 17:34:37 +0000

Hi guys. This time, we will talk about Pointers and Nil, which are related to one another, and why we must be careful when handling these two things in Go.

While it is wise to learn from experience,
it is wiser to learn from the experiences of others.
~ Rick Warren

Pointer

Pointer is a feature that is quite common in low-level languages such as C/C++. A pointer is an object/variable that stores a memory address, which later the object/variable can access the value of that memory address. Pointers, in simple terms, are where one variable references a value to another place (Pointing).

One of the uses of pointers is to save memory usage when using data that is quite large. For example, pass a large array to a function. However, pointers are dangerous because of their challenges, so several programming languages, such as Java, Python, Javascript, etc., remove this feature from being accessed directly by programmers, even though they are still used indirectly.

Nil

Nil or Null indicates the absence of value or that a variable has no value. Pointers are closely related to nil. In the Go programming language, the default value for pointers is nil.

Pointer in Go

In the Go, a pointer is explicitly indicated by the codebase's symbols * and &. The following is an explanation of these two symbols.

& (reference) is used to get the memory address of a variable
* (dereference) is used to get the variable's value pointed to by the pointer.



var1 := 12

// pointer1 refer to var1
pointer1 := &var1

fmt.Println(pointer1)
// Output: 0xc000012028

// Ask the pointer1, what is the value of the object you refer to
fmt.Println(*pointer1) 
// Output: 12

var1 = 33
fmt.Println(*pointer1)
// Output: 33

Memory management in the Go is not as broad as in other languages, such as C/C++, where the programmer must allocate and deallocate the program's memory. Garbage collector makes several challenges in the C/C++ language no longer exist in Go, for example, dangling pointers (pointers that point to memory addresses that have been cleared), memory leaks, and minimizing human error when programming compared to manual memory management.

Pointer Problems in Go

The pointer problem is closely related to the nil. One of the common problems in Go is the nil dereference error or, in other programming languages, usually known as Null Pointer Exception. This error occurs when a pointer asks the value of a memory address that refers to nil/nothing. This error will invoke panic error. Panic errors in Go can stop the entire running program. Here is an example.



package main

import "fmt"

func main() {
    var pointer1 *int // the value is nil
    fmt.Println(*pointer1)
}

// RESULT:
// panic: runtime error: invalid memory address or nil pointer dereference
// [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x480496]

// goroutine 1 [running]:
// main.main()
//  /tmp/sandbox751150285/prog.go:7 +0x16

// Program exited.

Before doing anything to the pointer variable, the variable must be checked whether the pointer is nil.



package main

import "fmt"

func main() {
    var pointer1 *int
    if pointer1 != nil {
        fmt.Println(*pointer1)
    }
}

This is relatively easy to do once we know we use pointers. This can be seen from using the symbols * and & in the codebase, which explicitly use pointers in Go.

The Dangers of Pointers in Go

Handling pointers that are used explicitly (where there are symbols * and & in the source code) is easy because they are visible directly when we are programming, and we can easily find these symbols in the codebase using a text editor.

The danger arises when there is implicit use of pointers where even the programmer is unaware that they are using pointers. The two symbols * and & are not shown in the codebase, so the nil check is often missed. This usually continues in code reviews, where reviewers are also often unaware because it requires high accuracy to detect the implicit use of pointers. For example, see the following code.



package main

import "fmt"

type UserInput struct {
    Name string
    Age  int
}

type UserRepository interface {
    SaveToDB(UserInput) error
}

type userService struct {
    userRepository UserRepository
}

func (s userService) Create(input UserInput) error {
    return s.userRepository.SaveToDB(input)
}

func main() {
    userService := userService{}
    userService.Create(UserInput{Name: "John", Age: 10})
}

// RESULT:
// panic: runtime error: invalid memory address or nil pointer dereference
// [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x45caf0]

// goroutine 1 [running]:
// main.userService.Create(...)
//  /tmp/sandbox2421441321/prog.go
// main.main()
//  /tmp/sandbox2421441321/prog.go +0x10

// Program exited.

In the code above, there is no explicit use of pointers, no * and & symbols in the source code. Where did the pointer error occur, and for what reason?

The error occurs in the source code above because the userService.userRepository value is nil. This is because when initializing, userService:= userService{} does not include values from userRepository, which has a type of interface. Interface in Go has a default value of nil. This error is not detected at compile time. A panic error will occur when the userService.Create() method is invoked.

This can be solved by creating a constructor function like the following source code.



package main

type UserInput struct {
    Name string
    Age  int
}

type UserRepository interface {
    SaveToDB(UserInput) error
}

type userRepository struct {
}

func (r userRepository) SaveToDB(input UserInput) error {
    return nil
}

type UserService interface {
    Create(UserInput) error
}

type userService struct {
    userRepository UserRepository
}

func (s userService) Create(input UserInput) error {
    return s.userRepository.SaveToDB(input)
}

func NewUserService(userRepository UserRepository) UserService {
    return &userService{userRepository: userRepository}
}

func main() {
    // initialize using the constructor
    userService := NewUserService(userRepository{})

    // call method
    userService.Create(UserInput{Name: "John", Age: 10})
}

The code above works well and can solve the previous problem. The code above is good if only a few interfaces are used. When using several interfaces, human error is possible, for example.



type transactionService struct {
   userRepository    UserRepository
   productRepository ProductRepository
   orderRepository   OrderRepository
   paymentRepository PaymentRepository
}

func (s transactionService) Create(input TransactionInput) error {
    user, _ := s.userRepository.Get(input.UserID)
    product, _ := s.productRepository.Get(input.ProductID)
    balance, _ := s.paymentRepository.GetBalance(input.UserID)
    totalPrice := input.Quantity * balance * product.Price
    _, err := s.orderRepository.Save(Order{
        UserID:     user.ID,
        ProductID:  input.ProductID,
        Quantity:   input.Quantity,
        TotalPrice: totalPrice,
    })
    return err
}

func NewTransactionServiceFail1(
    userRepository UserRepository,
    productRepository ProductRepository,
    orderRepository OrderRepository,
    paymentRepository PaymentRepository,
) TransactionService {
    return &transactionService{
        userRepository:    userRepository,
        productRepository: productRepository,
        orderRepository:   orderRepository,
        // MISTAKE 1: Forget to add a parameter here
    }
}

func NewTransactionServiceFail2(
   userRepository UserRepository,
   productRepository ProductRepository,
   paymentRepository PaymentRepository,
   // MISTAKE 2: Forget to add a parameter of an interface
) TransactionService {
   return &transactionService{
       userRepository:    userRepository,
       productRepository: productRepository,
       paymentRepository: paymentRepository,
   }
}

The two constructor functions will not error when initializing the struct but will error when the Create() method is called. This is because there is a nil field in the transactionService struct. A strict code review process is needed to prevent this, but this human error often occurs when many interfaces are used.

The solution to this problem is to check when the struct is initialized so we can detect errors earlier. The following is an example implementation of this solution.



func NewTransactionService(
    userRepository UserRepository,
    productRepository ProductRepository,
    orderRepository OrderRepository,
    paymentRepository PaymentRepository,
) TransactionService {
    svc := transactionService{
        userRepository:    userRepository,
        productRepository: productRepository,
        orderRepository:   orderRepository,
        paymentRepository: paymentRepository,
    }
    if !Valid(svc) {
        panic("please initialize correctly")
    }
    return &svc
}

func Valid(s any) bool {
    v := reflect.ValueOf(s)
    t := v.Type()

    if t.Kind() != reflect.Struct {
        return false
    }

    for i := 0; i < t.NumField(); i++ {
        field := v.Field(i)
        if field.Kind() == reflect.Interface && field.IsNil() {
            return false
        }
    }
    return true
}

Using this solution, the Valid function protects the struct so that no fields have a nil value. In this function, each field is looped and checked whether it is of the interface data type and has a nil value. Just remember to be careful when using functions that use reflection.

Another example of a nil interface is the error interface.



package main

import "fmt"

func main() {
    var err error
    fmt.Println(err.Error())
}

// RESULT:

// panic: runtime error: invalid memory address or nil pointer dereference
// [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x480490]

// goroutine 1 [running]:
// main.main()
//  /tmp/sandbox174934985/prog.go:7 +0x10

// Program exited.

Nil Default Value

In the Go programming language, data types have default values. For example, a string is "" (empty string), float and int = 0, and boolean = false. Several data types use nil as the default value: interface, pointer, slice, map, and channel. These data types are what we need to handle carefully.



package default_value

import (
   "testing"

   "github.com/stretchr/testify/require"
)

type I interface {
   M()
}

func TestDefaultValue1(t *testing.T) {
   var pointer *int
   var slice []int
   var m map[string]any
   var channel chan int
   var i I

   require.Nil(t, pointer)
   require.Nil(t, slice)
   require.Nil(t, m)
   require.Nil(t, channel)
   require.Nil(t, i)
}

// Test: Passed

Like the interface in the previous discussion, slice, map, and channel are examples of implicit use of pointers in Go because there is no use of & and * symbols. Of course, this makes us need to be more careful when using these three data types because it may trigger unusual behaviour.

Slice

Slices are a nullable data type in Go, unlike arrays. The slice itself has a default value of nil. Let's take a look at the following source code.



package main

import "fmt"

func main() {
    var s []int
    if s == nil {
      fmt.Println("it is nil", s)
    }
}

// Output: it is nil []

The slice has a nil value in that source code, but when printed, it will produce []. That's quite strange, isn't it?. Next, let's compare it with the source code below.



package main

import "fmt"

func main() {
    s := []int{} // using slice literal
    if s == nil {
        fmt.Println("it is nil", s)
    } else {
        fmt.Println("it is not nil", s)
    }
}

// Output: it is not nil []

Nil slice and Empty slice are represented similarly in Go to reduce ambiguity and consistency in how slices are handled in the Go programming language.

This is where we need to be careful because not all behaviour will be like this. One of them is the default Go encoding/json library that differentiates those two cases.



package main

import (
    "encoding/json"
    "fmt"
)

type Example struct {
    NilSlice   []int `json:"nil_slice"`
    EmptySlice []int `json:"empty_slice"`
}

func main() {
     var nilSlice []int
     emptySlice := []int{}

     example := Example{NilSlice: nilSlice, EmptySlice: emptySlice}
     bytes, _ := json.Marshal(example)
     fmt.Println(string(bytes))
}

// Output: {"nil_slice":null,"empty_slice":[]}

This won't be a problem in Go. Still, it will be problematic when it's used as an API by other applications that use different programming languages that differ in handling nil and empty arrays. I have experienced this case when migrating the backend to Go while the frontend Javascript application should not have changed. Several errors appeared on the front end due to not handling this case.

Map

Handle nil on the map is more tricky than the slice. We cannot use a map with a default value of nil because it will cause a panic error.



package main

import (
   "fmt"
)

func main() {
   var m map[string]any
   m["key"] = 12
   fmt.Println(m)
}

// RESULT: 
// panic: assignment to entry in nil map

// goroutine 1 [running]:
// main.main()
//  /tmp/sandbox3489622551/prog.go:9 +0x2c

// Program exited.

Therefore, we should always initialize the map using the map literal or the make function.



package main

import "fmt"

func main() {
    var m0 = map[int]int{} // map literanl
    var m1 = make(map[int]int) // make function
    m2 := map[int]int{} // map literal
    m3 := make(map[int]int) // make function

    m0[1] = 1
    m1[2] = 2
    m2[3] = 3
    m3[4] = 4

    fmt.Println(m0, m1, m2, m3)
}

// Output: map[1:1] map[2:2] map[3:3] map[4:4]

When we want to use multiple keys, we can use a nested map, which is also quite vulnerable to nil errors.



package main

import "fmt"

func main() {
    m := map[int]map[int]int{}
    m[1][1] = 1
    fmt.Println(m)
}

// RESULT:
// panic: assignment to entry in nil map

// goroutine 1 [running]:
// main.main()
//  /tmp/sandbox1619142114/prog.go:7 +0x45

// Program exited.

Even though it has been initialized as above, items in the nested map must also be initialized to avoid this error. Another way to prevent this is to use a combined key using a string when multiple keys are needed.



package main

import "fmt"

func main() {
    // first way (nested map)
    m := map[int]map[int]int{}
    m[1] = map[int]int{}
    m[1][1] = 1

    // second way (combination keys using a string)
    m1 := map[string]int{}
    m1[fmt.Sprintf("%v_%v", 1, 1)] = 1

    fmt.Println(m, m1)
}

// Output: map[1:map[1:1]] map[1_1:1]

Channel

Channels in Go are a feature to connect one goroutine with another goroutine, where this channel can be used to exchange information between goroutines. Channel is a data type that uses nil as the default value.

Channels with nil value cannot be used because they will cause the program to be deadlocked.



package main

import (
    "fmt"
)

func main() {
    var done chan bool

    values := []string{"a", "b", "c"}
    for _, v := range values {
        go func(v string) {
            fmt.Print(v)
            done <- true
        }(v)
    }

    for range values {
        <-done
    }

}

// RESULT:
// cab
// fatal error: all goroutines are asleep - deadlock!

// goroutine 1 [chan receive (nil chan)]:
// main.main()
//  /tmp/sandbox2242567277/prog.go:19 +0x149

// goroutine 6 [chan send (nil chan)]:
// main.main.func1({0x4b1d28?, 0x0?})
//  /tmp/sandbox2242567277/prog.go:14 +0x69
// created by main.main in goroutine 1
//  /tmp/sandbox2242567277/prog.go:12 +0x75

// goroutine 7 [chan send (nil chan)]:
// main.main.func1({0x4b11b8?, 0x0?})
//  /tmp/sandbox2242567277/prog.go:14 +0x69
// created by main.main in goroutine 1
//  /tmp/sandbox2242567277/prog.go:12 +0x75

// goroutine 8 [chan send (nil chan)]:
// main.main.func1({0x493458?, 0x0?})
//  /tmp/sandbox2242567277/prog.go:14 +0x69
// created by main.main in goroutine 1
//  /tmp/sandbox2242567277/prog.go:12 +0x75

// Program exited.

So, always use the make() function when initializing channels.



package main

import (

    "fmt"

)

func main() {

    done := make(chan bool)

<span class="n">values</span> <span class="o">:=</span> <span class="p">[]</span><span class="kt">string</span><span class="p">{</span><span class="s">"a"</span><span class="p">,</span> <span class="s">"b"</span><span class="p">,</span> <span class="s">"c"</span><span class="p">}</span>
<span class="k">for</span> <span class="n">_</span><span class="p">,</span> <span class="n">v</span> <span class="o">:=</span> <span class="k">range</span> <span class="n">values</span> <span class="p">{</span>
    <span class="k">go</span> <span class="k">func</span><span class="p">(</span><span class="n">v</span> <span class="kt">string</span><span class="p">)</span> <span class="p">{</span>
        <span class="n">fmt</span><span class="o">.</span><span class="n">Print</span><span class="p">(</span><span class="n">v</span><span class="p">)</span>
        <span class="n">done</span> <span class="o">&lt;-</span> <span class="no">true</span>
    <span class="p">}(</span><span class="n">v</span><span class="p">)</span>
<span class="p">}</span>

<span class="k">for</span> <span class="k">range</span> <span class="n">values</span> <span class="p">{</span>
    <span class="o">&lt;-</span><span class="n">done</span>
<span class="p">}</span>



}

// Output: cab

Pass By Reference

Function parameters with data types that use nil as a default value do not indicate explicit use of pointers (the symbols * and &), so we have to check manually.



package main

import (

   "errors"

   "fmt"

)

type I interface {

    M()

}

type i struct{}

func (i i) M() {}

func fn(i I, ch chan int, s []int, m map[int]int) error {

    fmt.Println(i, ch, s, m)

    return nil

}

// manual check for safety

func fn1(i I, ch chan int, s []int, m map[int]int) error {

    if i == nil {

        return errors.New("i interface nil")

    }

    if ch == nil {

        return errors.New("ch channel nil")

    }

    if s == nil {

        return errors.New("s slice nil")

    }

    if m == nil {

        return errors.New("m map nil")

    }

    fmt.Println(i, ch, s, m)

    return nil

}

func main() {

    fn(nil, nil, nil, nil)

    fn1(i{}, make(chan int), []int{}, map[int]int{})

}

// OUTPUT:

// <nil> <nil> [] map[]

// {} 0xc000076060 [] map[]

Summary

Handling related to pointers and nil in the Go programming language requires precision and caution so undesirable behaviour, such as panic or deadlock, does not occur. Explicit pointers can be checked quickly, but implicit pointers are more challenging.

When using a data type with a default value of nil. The general rule I usually use when initializing variables is using short variable declaration (using the := operator), which forces us to provide a value before the variable is used. Uses the make() function and slice literal ([]int{}) and map literal (map[int]int{}), and prefers to use a map with string keys when more than one combined key/key is needed.

Improved code review is critical as an initial filter before merging the code with the main codebase.

That's all from me. I hope you get useful insight.

Thank You.

Generic in Golang: Write utility functions easier

Laba Subagia — Sat, 03 Feb 2024 12:08:29 +0000

In this article, I will demonstrate why it is easier to write utility functions/methods using Generic in Golang. The utility function itself is a function that performs common behavior and can be reused in several places.

Generic programming is a style of computer programming in which algorithms are written in terms of types to-be-specified-later that are then instantiated when needed for specific types provided as parameters.

By using Generic in Go, we can have a codebase with less duplication because my codebase had several duplications before I used Generic. Especially the codebase that uses Go version 1.17 and below.

“Talk is cheap. Show me the code.” ― Linus Torvalds

func InArray(haystack []string, needle string) bool {
  for _, hay := range haystack {
    if hay == needle {
       return true
    }
  }
  return false
}

The InArray function aims to check whether an item exists in an array. In practice, I need to duplicate the function if I check arrays with another data type.

func InArrayString(haystack []string, needle string) bool {
  for _, hay := range haystack {
    if hay == needle {
       return true
    }
  }
  return false
}

func InArrayInt(haystack []int, needle int) bool {
  for _, hay := range haystack {
    if hay == needle {
      return true
    }
  }
  return false
}

func InArrayFloat(haystack []float64, needle float64) bool {
  for _, hay := range haystack {
    if hay == needle {
      return true
    }
  }
  return false
}

If we want to remove code duplication from the codebase without using generic, we can use reflect library, especially if the codebase still uses Golang with version 1.17 or below.

func InArray(haystack any, needle any) (bool, error) {
  haystackValue := reflect.ValueOf(haystack)
  if haystackValue.Kind() != reflect.Slice && haystackValue.Kind() != reflect.Array {
     return false, errors.New("haystack is not an array")
  }

  needleValue := reflect.ValueOf(needle)
  for i := 0; i < haystackValue.Len(); i++ {
     val := haystackValue.Index(i)
     if val.Kind() == needleValue.Kind() && val.Interface() == needleValue.Interface() {
       return true, nil
     }
  }

  return false, nil
}

Using reflect can remove the duplication in the code, but, It will increase the complexity of the function. But with a cost, we somehow lose the "static typing" ability because it needs to check the data type at runtime. Besides that, we need to ensure the input provided is valid and handle errors when we use the function.

If you still want to avoid adding more complexity to your code, you can convert the array first and then pass it to the utility function.

// Declaration
func InArray(haystack []string, needle string) bool {
  for _, hay := range haystack {
    if hay == needle {
       return true
    }
  }
  return false
}


func main() {

  // Usage
  haystack := []int{1, 2, 3, 4, 5}
  needle := 1

  haystackString := make([]string, 0, len(haystack))
  for _, hay := range haystack {
     haystackString = append(haystackString, string(hay))
  }

  fmt.Println(InArray(haystackString, string(needle)))
}

With conversion, you need to convert your array every time your array type is different from the InArray argument. In other words, you will still have duplicate code when you use that utility function. Besides that, the conversion adds more complexity to your code (time complexity in the code above).

We can remove duplication and still have a simple code using Generic.

func InArray[T comparable](haystack []T, needle T) bool {
  for _, hay := range haystack {
    if hay == needle {
      return true
    }
  }
  return false
}

The code with generic and without generic at first is similar, and you don't have to add more complexity like when using reflect library.

Generic is suitable for making a utility function with more than one input type. But don't overuse it.

OK, that's all. You can also play with the code here. Thank you.

A Journey to Find an Ultimate Development Environment

Laba Subagia — Fri, 02 Feb 2024 16:38:20 +0000

This is my journey of finding the ultimate development environment for my personal needs.

The Criteria

I have several criteria for the ultimate development environment that I dream of. These are they.

Easy to set up or at least follow a certain rule to do it.
Don't waste time.
Operating System (OS) agnostic
"Work On My Machine" not anymore.
Handle dependency versioning and always up-to-date.
Ability to develop anywhere in no time (code local or online).
Minimal Resource usage.
Text Editor / IDE Compatability for local development.

DISCLAIMER: This is only tested in Linux because I used Linux for development. The arguments also can be very subjective.

OK, let's get started.

The Traditional Approach: install each dependency

A traditional way to create a development environment is to install and configure each tool one by one.

An example of this is XAMPP, LAMPP, or installing PHP, MySQL, and other dependency separately. Things that I really hate about this method is has a high human error possibility.

In this way, your company usually will have a document or checklist that instructs step by step how to set up your local development environment and even how to set up each deployment to make sure not a single step is overlooked.

Let's compare this approach to the criteria above.

❌ Hard to set up and it usually needs a checklist for it (maybe this criteria is partially incorrect if there is a packaged thing like an XAMPP installer).
❌ Waste of time to install everything one by one and make sure all of them can work together.
❌ Not OS agostic (Windows, Mac, Linux), even in Linux you'll have different configurations of each distro based on their package manager (APT, Yum, Pacman, Zypper), or the worst, you'll need to deal with Dependency Hell in distro like Slackware.
❌ Error-prone, need to make sure that every modified config in local development needs to be updated on the deployment server. This is hard.
❌ When you need two different versions of a dependency, that will need more configuration (especially Linux).
❌ You need to configure it everywhere.
✅ Resource usage is minimal because you only use what you need.
✅ Fully compatible with IDE/Text Editor.

The Virtual Machine (VM) Approach

Using a virtual machine as an isolated development environment is a good way to keep your host OS clean and easier. when you have trouble configuring your development environment you just reset it using snapshot. You can also use different versions of dependency in each project in different VMs.

Although it offers several benefits. For me, this is my least favorite way to do development until this day. Because it is resource-heavy in CPU, RAM, and Storage you need to develop it inside the VM or need a separate configuration to be able to develop using Host OS.

Let's compare this approach to the criteria above.

❌ Hard to set up tools inside the VM, especially if you also install all tools one by one.
❌ Waste of time to install everything one by one and make sure all of them can work together inside the VM. There is also need time to install the Guest OS.
❌ Not OS agostic for installing each tool in VM.
✅ to deploy, you can just export the VM or use the snapshot.
🟡 Still hard inside VM, but if you decide to use a different VM, that is easy.
🟡 It will work on every machine, but it may take time to install the VM. To develop everywhere you'll maybe need to configure SSH or other Remote Development tools manually.
❌ This is resource-heavy, even if it is Type-1 Virtualization. (CPU, RAM, Storage). This is the worst among all approaches.
🟡 Guest OS is compatible with IDE/Text Editor. But to develop with Host OS is a bit challenging, maybe need to use Remote Development or SSH.

The Version Manager Approach

The purpose of a version manager is to help you navigate or install any tools for development easily. Version Manager can be one tool for each dependency (e.g. NVM, g) or One tool for all dependencies (e.g. asdf, mise).

This is maybe the way that I use the longest time among the others, it is pretty easy to set up especially when you are using a popular OS (e.g. Ubuntu). These tools might be helpful but it's not perfect. There are some corner cases I've experienced e.g. sometimes you'll need to install several dependencies manually (e.g. install PHP), or you'll need to build from scratch when you use a more niche OS (e.g. NixOS).

Almost all of these tools require setup manually for each tool. Last time I checked, only Mise provided a declarative way to configure a development environment.

[env]
# supports arbitrary env vars so mise can be used like direnv/dotenv
NODE_ENV = 'production'

[tools]
# specify single or multiple versions
terraform = '1.0.0'
erlang = ['23.3', '24.0']
# supports everything you can do with .tool-versions currently
node = ['16', 'prefix:20', 'ref:master', 'path:~/.nodes/14']
# send arbitrary options to the plugin, passed as:
# MISE_TOOL_OPTS\_\_VENV=.venv
python = {version='3.10', virtualenv='.venv'}

[plugins]
# specify a custom repo url
# note this will only be used if the plugin does not already exist
python = 'https://github.com/asdf-community/asdf-python'

[alias.node] # project-local aliases
my_custom_node = '20'

Let's compare this approach to the criteria above.

✅ Easy to set up, even with some manual commands, but it's not annoying especially the version manager that provides a declarative way to set up a development environment.
✅ The setup is quite fast most of the time unless you hit the corner cases.
❌ Not really OS agnostic based on the corner cases.
🟡 You'll need to configure the server. Most of the time will work out of the box unless you hit the corner cases.
✅ Handle package versioning really well. You can use different packages in one machine and choose between the oldest and latest versions of that package. I think this is the best way of all.
❌ You'll need to configure the other machine too.
✅ Resource usage is minimal because you only use what you need.
✅ Fully compatible with IDE/Text Editor.

The Configuration Management Way

Configuration management is a process for maintaining computer systems, servers, applications, network devices, and other IT components in a desired state. It's a way to help ensure that a system performs as expected, even after many changes are made over time. ~RedHat

In this article's context, it is simply a tool that provides a declarative way to automate your machine/OS to configure the development machine as you want (install package, modify the configuration, etc). Examples of these tools are Ansible, Puppet, etc.

In My case, I use Ansible. Most of the time the config is easy especially when all you need is only install the package from the OS repository. But things can be a little bit challenging when you need to install from scratch or need additional configuration.
This is an example of a simple install OS package

---
- name: setup apt apps
  become: true
  apt:
    name:
      - build-essential
      - htop
      - tmux
      - exa
      - neovim
      - zsh
      - autojump
      - trash-cli
      - btop
      - fzf
      - tldr
    state: latest

This is an example of a more challenging config

---
# https://docs.docker.com/engine/install/ubuntu/
- name: setup docker
  become: true
  when: not is_wsl
  vars:
    apt_key_path: /etc/apt/keyrings/docker.asc
  block:
    - name: docker install prerequisite
      apt:
        name:
          - ca-certificates
          - curl
          - gnupg
        state: latest

    - name: docker get upstream distribution codename (ubuntu codename)
      when: is_mint
      shell: . /etc/upstream-release/lsb-release && echo "$DISTRIB_CODENAME"
      register: codename_mint

    - name: docker get distribution codename
      when: is_ubuntu
      shell: . /etc/os-release && echo "$VERSION_CODENAME"
      register: codename_ubuntu

    - name: docker get dpkg architecture
      shell: dpkg --print-architecture
      register: dpkg_arch

    - name: docker set facts
      set_fact:
        distribution_codename: '{{ codename_mint.stdout if is_mint else codename_ubuntu.stdout }}'
        dpkg_arch: '{{ dpkg_arch.stdout }}'

    - name: docker get apt-key
      get_url:
        url: https://download.docker.com/linux/ubuntu/gpg
        dest: '{{ apt_key_path }}'

    - name: docker add repository
      apt_repository:
        repo: deb [arch={{ dpkg_arch }} signed-by={{ apt_key_path }}] https://download.docker.com/linux/ubuntu {{ distribution_codename }} stable
        filename: docker
        state: present

    - name: docker install apps
      apt:
        name:
          - docker-ce
          - docker-ce-cli
          - containerd.io
          - docker-buildx-plugin
          - docker-compose-plugin
        state: latest

    - name: docker enable service
      service:
        name: docker
        state: started
        enabled: true

    - name: docker add group
      group:
        name: docker
        state: present

    - name: docker add user "{{ user }}" into docker group
      user:
        name: '{{ user }}'
        groups: docker
        append: true

As you can see when installing docker, you'll need more research about Ansible when doing more complicated tasks e.g. adding a repository, reading OS codename, etc.

I've even tried to automate an OS with Ansible. You can see it here.

Let's compare this approach to the criteria above.

🟡 setup can be easy or hard depending on what you want to accomplish.
✅ a lot of setup/research needs to be done upfront, but when it's already finished, you just need one command to execute all things. I think it's a good investment.
❌ This is not OS agnostic, e.g. different OS may have a different package manager and that also needs separate configurations.
✅ A good way to set up multiple machines.
❌ Can't handle versioning by default. Unless combined with version manager, etc.
🟡 To be able to develop anywhere, you'll need at least run Ansible script and configure SSH or Remote Dev tools.
✅ Resource usage is minimal because you only use what you need.
✅ Fully compatible with IDE/Text Editor Fully compatible with IDE/Text Editor.

The Nix Package

Nix is a package manager that makes reproducible, declarative, and reliable systems. ~ nixos.org

Nix / Nixpkgs is really good at building reproducible development environments that can be easily duplicated in other machines. Nix takes a functional programming approach where the same input should produce the same output. Nix also works well in other Linux distributions (outside NixOS) and can exist alongside default system package managers like apt and yum.

But so far, the best way to use Nix is by using NixOS itself, because you have a single source of truth to configure your entire system. This is why I admire NixOS so much and hopefully, other OS also take this approach in the future. Other than that, nix is also the largest repository so far with more than 80.000 packages.

Things that I don't really like about Nix is you'll need to learn their scripting language to configure your system. The learning curve is quite steep. Even though the number of packages is large, not all packages are always up to date, and being able to use the latest version requires a slightly challenging approach (e.g. override, unstable package, or directly contribute to Nixpkgs itself).

I also have a configuration in Nix, you can check it here.

Let's compare this approach to the criteria above.

🟡 setup is challenging in the beginning because you'll need to learn new scripting tools.
🟡 the learning curve is steep, but when you already know it. it is easier.
✅ Mostly so good with Linux and MacOS, but in Windows you can use WSL
✅ Work the same from one machine to another
✅ Versioning is limited to several latest versions, when you need to use older e.g. PHP5 or PHP4 it will not exist in the default repository. It is good enough, but not as good as using Version Manager.
🟡 It is possible to develop everywhere but you'll need to at least config nix first. One way to do it is to Run the Nix command directly on an online NixOS VM (or install Nix first on other OS VM). Then you can configure SSH or remote development tools.
✅ Resource is minimal, you only use what you need. There is also nix-collect-garbage to clean up your system.
✅ Fully compatible with IDE/Text Editor.

The Containerization

Containerization is a software deployment process that bundles an application's code with all the files and libraries it needs to run on any infrastructure. Containerization is similiar to a virtual machine except a virtual machine requires an operating system within than bundle.

By using containerization, the application will always have the same configuration that is used in the development environment and production environment. There is no more "It works on my machine". Some examples of containerization technologies are Docker and Podman.

There are several ways I use containerization/containers in the development environment.

Partial Usage

The first way to use it is to use containerization on several dependencies e.g. database, log, etc. You'll do the development process outside the containers but use the dependency needed from it.

An example of this is using docker-compose to spin up PostgreSQL, Grafana, Adminer, and Jaeger. But Golang is still installed on the OS. The reason for this is to enable full support of any tools that are required by text editor / IDE. Some IDEs might not be able to fully support all the language features when all the dependencies are inside the container.

I've used this approach for quite a while. The advantage of using this approach it can save resource usage when you need to work with multiple projects simultaneously. This is an example of how I structure the folder.

The same docker dependency is used by several projects at the same timeUsing one shared package in one docker-compose file that can be used on several projects (e.g. bind all ports to hosts). The drawback of this approach is you'll not be able to push the compose file to version control along with the project or you'll need to copy it to every project (unless you use mono repo).

One example of my repo that uses this approach is here. In this case, Include the docker-compose in the project repo.

Full Usage

The full usage of the container means that you'll do the development inside the container. All the tools for development need to be installed inside the container. One of the technologies that leverage this approach is Devcontainers.

Develop inside a containerThe advantage of this approach is not require any configuration related to the application (except Text Editor and Containerization Tool) in the OS. Several text editors such as VSCode or JetBrains already support doing development inside a container seamlessly.

When you push your code to Github, you can develop the app using codespace and it will automatically set up an online development environment for you. Other tools will make your life easier when developing using a dev container e.g. DevPod.

One disadvantage that I've experienced is when I need to go back and forth between several projects at the same time. Using this approach requires more computer resources to run all dependencies. An example of this is when ProjectA and ProjectB use PostgreSQL, by using this there will be two PostgreSQL instances running.

There is a workaround for cases like this by running PostgreSQL (or any dependencies) using separate docker-compose outside those two projects. However, if you have a high-end PC or are comfortable with that, it's fine.

There is an example of my project that uses Devcontainers here

Based on partial usage and full usage, my assessment regarding the use of containers as a development environment is as follows.

✅ It is easy to set up the development environment when you already know the docker.
✅ The initial setup may take some time, but subsequent setups will be automatic or require fewer commands.
✅ This approach is OS agnostic.
✅ It works on every machine.
✅ You can use any operating system and dependency in a Docker container. It is isolated and up to date, preventing conflicts.
✅ Develop your code from anywhere using tools like DevPod, Codespace, GitPod, and so on.
🟡 It's important to keep in mind that resource usage, such as CPU, RAM, and storage usage, may be high when running the same dependency, such as PostgreSQL, in multiple instances for different projects. Additionally, it's important to regularly monitor storage usage for containerization, as the more projects you have, the more disk space it will consume. If you have a high-end computer, this shouldn't be a problem.
🟡 Not all text editors/IDE support developing inside the container (e.g. Helix, NeoVIM (maybe supported via plugin), Nano, etc.). But if you use partial usage of containerization, this shouldn't be an issue.

Combination Way

All the approaches above are not exclusive to each other. In my Software Engineering Career, I've combined several approaches above to get the best out of it. Some examples are:

Use NixOS with declarative configuration along with docker in almost all of my projects. The Nixpkgs are used to configure the OS as a whole and Install the programming language. Partial Usage of docker for dependencies like Database, Log, and Monitoring dashboard.
Using Version Manager along with Configuration Management when I use LinuxMint. You can check it here.
Partial Usage of Containerization itself is also an example combination of a traditional way to install programming language and its tool in the OS using the default package manager for better text editor support and run dependencies like Database using Docker.

You can utilize all the approaches above to satisfy your needs.

Honorable Mention: Google Colab

If you ask me: "Is there any tool or development environment that satisfies all your criteria or at least almost all of it?". My answer will be Google Colab.

I encountered this when I was learning about Machine Learning and all related to that stuff. One tool that is always recommended everywhere is Google Colab. It's a free development environment that you can access anywhere. This tool also can be used to develop real projects. Even when I don't have GPU on my machine, Google Collab already provides it.

All you need to have before to be able to access Google Colab is a Web Browser and a Google Account (I think everyone already have it). This is the best development experience that I have had so far. Currently, Devcontainers is the one that is close to this (especially with Github Codespace).

Summary

It's a quite long journey so far, sometimes I still exploring other tools or other approaches to improve my development workflow. For now, I mostly use Devcontainer for my primary projects and use Version Manager or default package manager for small side projects. For side projects, I don't want to use too much storage for now, because of storage limitations.

That's all for me, right now. Maybe I will update this article when I discover some interesting tools or approaches to improve development workflow.

I hope you get something from this article. Forgive any grammar mistakes (English is not my native language).

Thank you.