DEV Community: St. B

Iterium - Generic Channel-based Iterators for Golang

St. B — Fri, 15 Sep 2023 11:38:31 +0000

The Iterium package is a powerful toolkit for creating and manipulating generic iterators in Golang. Inspired by the popular Python itertools library, Iterium provides a variety of functions for working with iterators in different ways.

- - -
If you like the project or idea, I'd appreciate your support on github in the form of an ⭐️ Star or a new PR to improve the code.

🌍 Github: https://github.com/mowshon/iterium
- - -

Iterium is designed to be easy to use and flexible, with a clear and concise API that enables you to create iterators that meet your specific needs. Whether you're working with strings, arrays, slices, or any other data type. Iterium makes it easy to traverse, filter, and manipulate your data with ease.

Decrypting the MD5 hash in Golang
- Benchmark
Iterator architecture
Creating an Iterator
Getting data from an iterator
Combinatoric iterators
- 🟢 Product() - Cartesian Product
- 🟢 Permutations()
- 🟢 Combinations()
- 🟢 CombinationsWithReplacement()
Infinite iterators
- 🔴 Count()
- 🔴 Cycle()
- 🔴 Repeat()
Finite iterators
- 🔵 Range()
- 🔵 Map()
- 🔵 StarMap()
- 🔵 Filter()
- 🔵 FilterFalse()
- 🔵 Accumulate()
- 🔵 TakeWhile()
- 🔵 DropWhile()
Create your own iterator

Installation

go get github.com/mowshon/iterium@v1.0.0

Import

import "github.com/mowshon/iterium"

Iterium provides a powerful set of tools for fast and easy data processing and transformations.

Decrypting the MD5 hash in Golang

Before we move on to explore each iterator in particular, let me give you a small example of decrypting an md5 hash in a few lines of code using Iterium. Assume that our password consists only of lower-case Latin letters and we don't know exactly its length, but assume no more than 6 characters.

// result of md5("qwerty") = d8578edf8458ce06fbc5bb76a58c5ca4
passHash := "d8578edf8458ce06fbc5bb76a58c5ca4"

for passLength := range Range(1, 7).Chan() {
    fmt.Println("Password Length:", passLength)

    // Merge a slide into a string.
    // []string{"a", "b", "c"} => "abc"
    join := func(product []string) string {
        return strings.Join(product, "")
    }

    // Check the hash of a raw password with an unknown hash.
    sameHash := func(rawPassword string) bool {
        hash := md5.Sum([]byte(rawPassword))
        return hex.EncodeToString(hash[:]) == passHash
    }

    // Combine iterators to achieve the goal...
    decrypt := FirstTrue(Map(Product(AsciiLowercase, passLength), join), sameHash)

    if result, err := decrypt.Next(); err == nil {
        fmt.Println("Raw password:", result)
        break
    }
}

Output:

Raw password: qwerty

Let's look at what's going on here. The main thing we are interested in is the line:

decrypt := FirstTrue(Map(Product(ascii, passLength), join), sameHash)

Initially the Product iterator generates all possible combinations of Latin letters from a certain length, and returns a slice like []string{"p", "a", "s", "s"}. AsciiLowercase is a slice of all lowercase Latin letters.;
Sending Product iterator to Map iterator which will use a closure-function to merge the slice into a string, like []string{"a", "b"} => "ab";
Sending the obtained iterator from Map to the FirstTrue iterator, which returns the first value from Map that returned true after applying the sameHash() function to it;
The sameHash() function turns the received string from the Map iterator into an md5 hash and checks if it matches with the unknown hash.

Benchmark ⏰

One of the special features of this package (compared to the python module) is the ability to know the exact number of combinations before running the process.

Product([]string{"a", "b", "c", "d"}, 10).Count() # 1048576 possible combinations

🔑 How many total combinations of possible passwords did it take to crack a 6-character md5 hash?

Password Length: 1, total combinations: 26
Password Length: 2, total combinations: 676
Password Length: 3, total combinations: 17576
Password Length: 4, total combinations: 456976
Password Length: 5, total combinations: 11881376
Password Length: 6, total combinations: 308915776

goos: linux
goarch: amd64
pkg: github.com/mowshon/iterium
cpu: AMD Ryzen 5 3600 6-Core Processor              
BenchmarkDecryptMD5Hash

Raw password: qwerty
BenchmarkDecryptMD5Hash-12             1  254100234180 ns/op

The hash was cracked in 4.23 minutes. This is just using the capabilities of the iterium package.

Iterator architecture

Each iterator corresponds to the following interface:

// Iter is the iterator interface with all the necessary methods.
type Iter[T any] interface {
    IsInfinite() bool
    SetInfinite(bool)
    Next() (T, error)
    Chan() chan T
    Close()
    Slice() ([]T, error)
    Count() int64
}

Description of the methods:

IsInfinite() returns the iterator infinite state;
SetInfinite() update the infinity state of the iterator;
Chan() returns the iterator channel;
Next() returns the next value or error from the iterator channel;
Close() closes the iterator channel;
Count() returns the number of possible values the iterator can return;
Slice() turns the iterator into a slice of values;

Creating an Iterator

You can use the function iterium.New(1, 2, 3) or iterium.New("a", "b", "c") to create a new iterator.

package main

import (
    "github.com/mowshon/iterium"
)

type Store struct {
    price float64
}

func main() {
    iterOfInt := iterium.New(1, 2, 3)
    iterOfString := iterium.New("A", "B", "C")
    iterOfStruct := iterium.New(Store{10.5}, Store{5.1}, Store{0.15})
    iterOfFunc := iterium.New(
        func(x int) int {return x + 1},
        func(y int) int {return y * 2},
        func(z int) int {return z / 3},
    )
}

Getting data from an iterator

There are two ways to retrieve data from an iterator. The first way is to use the Next() method or read values from the iterator channel range iter.Chan().

Using the Next() method:

func main() {
    iterOfInt := iterium.New(1, 2, 3)

    for {
        value, err := iterOfInt.Next()
        if err != nil {
            break
        }

        fmt.Println(value)
    }
}

Reading from the channel:

func main() {
    iterOfInt := iterium.New(1, 2, 3)

    for value := range iterOfInt.Chan() {
        fmt.Println(value)
    }
}

Combinatoric iterators

Combinatoric iterators are a powerful tool for solving problems that involve generating all possible combinations or permutations of a slice of elements, and are widely used in a range of fields and applications.

🟢 iterium.Product([]T, length) - Cartesian Product

The iterator generates a Cartesian product depending on the submitted slice of values and the required length. The Cartesian product is a mathematical concept that refers to the set of all possible ordered pairs formed by taking one element from each of two sets.

In the case of iterium.Product(), the Cartesian product is formed by taking one element from each of the input slice.

product := iterium.Product([]string{"A", "B", "C", "D"}, 2)
toSlice, _ := product.Slice()

fmt.Println("Total:", product.Count())
fmt.Println(toSlice)

Output:

Total: 16

[
    [A, A] [A, B] [A, C] [A, D] [B, A] [B, B] [B, C] [B, D]
    [C, A] [C, B] [C, C] [C, D] [D, A] [D, B] [D, C] [D, D]
]

🟢 iterium.Permutations([]T, length)

Permutations() returns an iterator that generates all possible permutations of a given slice. A permutation is an arrangement of elements in a specific order, where each arrangement is different from all others.

permutations := iterium.Permutations([]string{"A", "B", "C", "D"}, 2)
toSlice, _ := permutations.Slice()

fmt.Println("Total:", permutations.Count())
fmt.Println(toSlice)

Result:

Total: 12

[
    [A, B] [A, C] [A, D] [B, A] [B, C] [B, D]
    [C, B] [C, A] [C, D] [D, B] [D, C] [D, A]
]

🟢 iterium.Combinations([]T, length)

Combinations() returns an iterator that generates all possible combinations of a given length from a given slice. A combination is a selection of items from a slice, such that the order in which the items are selected does not matter.

combinations := iterium.Combinations([]string{"A", "B", "C", "D"}, 2)
toSlice, _ := combinations.Slice()

fmt.Println("Total:", combinations.Count())
fmt.Println(toSlice)

Output:

Total: 6

[
    [A, B] [A, C] [A, D] [B, C] [B, D] [C, D]
]

🟢 iterium.CombinationsWithReplacement([]T, length)

CombinationsWithReplacement() generates all possible combinations of a given slice, including the repeated elements.

result := iterium.CombinationsWithReplacement([]string{"A", "B", "C", "D"}, 2)
toSlice, _ := result.Slice()

fmt.Println("Total:", result.Count())
fmt.Println(toSlice)

Output:

Total: 10

[
    [A, A] [A, B] [A, C] [A, D] [B, B]
    [B, C] [B, D] [C, C] [C, D] [D, D]
]

Infinite iterators

Infinite iterators are a type of iterator that generate an endless sequence of values, without ever reaching an endpoint. Unlike finite iterators, which generate a fixed number of values based on the size of a given iterable data structure, infinite iterators continue to generate values indefinitely, until they are stopped or interrupted.

🔴 iterium.Count(start, step)

Count() returns an iterator that generates an infinite stream of values, starting from a specified number and incrementing by a specified step.

stream := iterium.Count(0, 3)

// Retrieve the first 5 values from the iterator.
for i := 0; i <= 5; i++ {
    value, err := stream.Next()
    if err != nil {
        break
    }

    fmt.Println(value)
}

stream.Close()

Output:

0, 3, 6, 9, 12, 15

🔴 iterium.Cycle(Iterator)

Cycle() returns an iterator that cycles endlessly through an iterator. Note that since iterium.Cycle() generates an infinite stream of values, you should be careful not to use it in situations where you do not want to generate an infinite loop. Also, if the iterator passed to Cycle() is empty, the iterator will not generate any values and will immediately close the channel.

cycle := iterium.Cycle(iterium.Range(3))

for i := 0; i <= 11; i++ {
    value, err := cycle.Next()
    if err != nil {
        break
    }

    fmt.Print(value, ", ")
}

Output:

0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2

🔴 iterium.Repeat(value, n)

Repeat() returns an iterator that repeats a specified value infinitely n = -1 or a specified number of times n = 50.

Here's an example code snippet that demonstrates how to use iterium.Repeat():

type User struct {
    Username string
}

func main() {
    // To receive an infinite iterator, you 
    // need to specify a length of -1
    users := iterium.Repeat(User{"mowshon"}, 3)
    slice, _ := users.Slice()

    fmt.Println(slice)
    fmt.Println(slice[1].Username)
}

Output:

[ User{mowshon}, User{mowshon}, User{mowshon} ]
mowshon

Finite iterators

Finite iterators return iterators that terminate as soon as any of the input sequences they iterate over are exhausted.

🔵 iterium.Range(start, stop, step)

Range() generates a sequence of numbers. It takes up to three arguments:

iterium.Range(end) # starts from 0 to the end with step = +1
iterium.Range(start, end) # step is +1
iterium.Range(start, end, step)

start: (optional) Starting number of the sequence. Defaults to 0 if not provided.
stop: (required) Ending number of the sequence.
step: (optional) Step size of the sequence. Defaults to 1 if not provided.

Here's an example code snippet that demonstrates how to use iterium.Range():

first, _ := iterium.Range(5).Slice()
second, _ := iterium.Range(-5).Slice()
third, _ := iterium.Range(0, 10, 2).Slice()
float, _ := iterium.Range(0.0, 10.0, 1.5).Slice()

fmt.Println(first)
fmt.Println(second)
fmt.Println(third)
fmt.Println(float)

Output:

first:  [0, 1, 2, 3, 4]
second: [0, -1, -2, -3, -4]
third:  [0, 2, 4, 6, 8]

float:  [0.0, 1.5, 3.0, 4.5, 6.0, 7.5, 9.0]

Features 🔥

Note that compared to range() from Python, Range() from Iterium if it receives the first parameter below zero, the step automatically becomes -1 and starts with 0. In Python such parameters will return an empty array.
Also, this iterator is more like numpy.arange() as it can handle the float type.

🔵 iterium.Map(iter, func)

Map() is a function that takes two arguments, a function and another iterator, and returns a new iterator that applies the function to each element of the source iterator, producing the resulting values one at a time.

Calculating the Fibonacci Number with Iterium

Here is an example code snippet that demonstrates how to use iterium.Map() to apply a function to each element from another iterator:

numbers := iterium.Range(30)
fibonacci := iterium.Map(numbers, func(n int) int {
    f := make([]int, n+1, n+2)
    if n < 2 {
        f = f[0:2]
    }

    f[0] = 0
    f[1] = 1

    for i := 2; i <= n; i++ {
        f[i] = f[i-1] + f[i-2]
    }

    return f[n]
})

slice, _ := fibonacci.Slice()
fmt.Println(slice)

Output:

[
    0 1 1 2 3 5 8 13 21 34 55 89
    144 233 377 610 987 1597 2584
    4181 6765 10946 17711 28657 46368
    75025 121393 196418 317811 514229
]

🔵 iterium.StarMap(iter, func)

StarMap() takes an iterator of slices and a function as input, and returns an iterator that applies the function to each slice in the iterator, unpacking the slices as function arguments.

Here's an example code snippet that demonstrates how to use iterium.StarMap():

func pow(a, b float64) float64 {
    return math.Pow(a, b)
}

func main() {
    values := iterium.New([]float64{2, 5}, []float64{3, 2}, []float64{10, 3})
    starmap := iterium.StarMap(values, pow)

    slice, _ := starmap.Slice()
    fmt.Println(slice)
}

Output:

[32, 9, 1000]

Note that iterium.StarMap() is similar to iterium.Map(), but is used when the function to be applied expects two arguments, unlike Map() where the function only takes in a single argument.

🔵 iterium.Filter(iter, func)

Filter() is used to filter out elements from an iterator based on a given condition. It returns a new iterator with only the elements that satisfy the condition.

Here is an example of using the iterium.Filter() function to filter out even numbers from a list:

func even(x int) bool {
    return x % 2 == 0
}

func main() {
    numbers := iterium.New(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
    filter := iterium.Filter(numbers, even)

    slice, _ := filter.Slice()
    fmt.Println(slice)
}

Output:

[2, 4, 6, 8, 10]

🔵 iterium.FilterFalse(iter, func)

FilterFalse() returns an iterator that contains only the elements from the input iterator for which the given function returns False.

Here is an example of using the iterium.FilterFalse() function to filter out even numbers from a list:

func even(x int) bool {
    return x % 2 == 0
}

func main() {
    numbers := iterium.New(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
    filter := iterium.FilterFalse(numbers, even)

    slice, _ := filter.Slice()
    fmt.Println(slice)
}

Output:

[1, 3, 5, 7, 9]

🔵 iterium.Accumulate(iter, func)

Accumulate() generates a sequence of accumulated values from an iterator. The function takes two arguments: the iterator and the function that defines how to combine the iterator elements.

Here's an example:

func sum(x, y int) int {
    return x + y
}

func main() {
    numbers := iterium.New(1, 2, 3, 4, 5)
    filter := iterium.Accumulate(numbers, sum)

    slice, _ := filter.Slice()
    fmt.Println(slice)
}

In this example, Accumulate() generates an iterator that outputs the accumulated sum of elements from the iterator numbers.

Output:

[1 3 6 10 15]

It also works fine with strings:

func merge(first, second string) string {
    return fmt.Sprintf("%s-%s", first, second)
}

func main() {
    letters := iterium.New("A", "B", "C", "D")
    filter := iterium.Accumulate(letters, merge)

    slice, _ := filter.Slice()
    fmt.Println(slice)
}

Output:

["A", "A-B", "A-B-C", "A-B-C-D"]

🔵 iterium.TakeWhile(iter, func)

TakeWhile() returns an iterator that generates elements from an iterator while a given predicate function holds true. Once the predicate function returns false for an element, TakeWhile() stops generating elements.

The function takes two arguments: an iterator and a predicate function. The predicate function should take one argument and return a boolean value.

Here's an example:

func lessThenSix(x int) bool {
    return x < 6
}

func main() {
    numbers := iterium.New(1, 3, 5, 7, 9, 2, 4, 6, 8)
    filter := iterium.TakeWhile(numbers, lessThenSix)

    slice, _ := filter.Slice()
    fmt.Println(slice)
}

Output:

[1, 3, 5]

In this example, TakeWhile() generates an iterator that yields elements from the numbers iterator while they are less than 6. Once TakeWhile() encounters an element that does not satisfy the predicate (in this case, the number 7), it stops generating elements.

Note that TakeWhile() does not apply the predicate function to all elements from the iterator, but only until the first element that fails the condition. In other words, TakeWhile() returns an iterator with values satisfying the condition up to a certain point.

🔵 iterium.DropWhile(iter, func)

DropWhile returns an iterator that generates elements from an iterator after a given predicate function no longer holds true. Once the predicate function returns false for an element, DropWhile starts generating all the remaining elements.

The function takes two arguments: an iterator and predicate function. The predicate function should take one argument and return a boolean value.

Here's an example:

func lessThenSix(x int) bool {
    return x < 6
}

func main() {
    numbers := iterium.New(1, 3, 5, 7, 9, 2, 4, 6, 8)
    filter := iterium.DropWhile(numbers, lessThenSix)

    slice, _ := filter.Slice()
    fmt.Println(slice)
}

Output:

[7, 9, 2, 4, 6, 8]

In this example, DropWhile() generates an iterator that yields elements from the numbers iterator after the first element that is greater than or equal to 6.

Note that DropWhile() applies the predicate function to all elements from the iterator until it finds the first element that fails the condition. Once that happens, it starts generating all the remaining elements from the iterator, regardless of whether they satisfy the predicate function.

DropWhile() is often used to skip over elements in an iterator that do not satisfy a certain condition, and start processing or generating elements once the condition is met.

Create your own iterator 🛠️

You can create your own iterators for your unique tasks. Below is an example of how to do this:

// CustomStuttering is a custom iterator that repeats
// elements from the iterator 3 times.
func CustomStuttering[T any](iterable iterium.Iter[T]) iterium.Iter[T] {
    total := iterable.Count() * 3
    iter := iterium.Instance[T](total, false)

    go func() {
        defer iter.Close()
        defer iterium.IterRecover()

        for {
            // Here will be the logic of your iterator...
            next, err := iterable.Next()
            if err != nil {
                return
            }

            // Send each value from the iterator
            // three times to a new channel.
            iter.Chan() <- next
            iter.Chan() <- next
            iter.Chan() <- next
        }
    }()

    return iter
}

func main() {
    numbers := iterium.New(1, 2, 3)
    custom := CustomStuttering(numbers)

    slice, _ := custom.Slice()
    fmt.Println(slice)
    fmt.Println("Total:", custom.Count())
}

Output:

[1 1 1 2 2 2 3 3 3]
Total: 9

Go: Insert a value into nested structures with a dot

St. B — Fri, 15 Sep 2023 09:42:53 +0000

This package was created to solve the problem of adding data in nested structures, maps, slices and channels - of ⚡️ ANY complexity and different data types. We know the exact path to the required field, but if there is a map on this path, we must first initialise the map correctly, check if such a key exists and then insert the value.

But, yes… if you have a simple structure, you don't need the dot package, but if you have a more complicated hierarchy, I'm happy to present my latest open source Golang project! 😎

Connect.the.Dots, Control.your.Data!

- - -
If you like the project or idea, I'd appreciate your support on github in the form of an ⭐️ Star or a new PR to improve the code.

🌍 Github: https://github.com/mowshon/dot
- - -

Below I will give a simple example of a problem I encountered. But, it is worth remembering that in my case I was talking about a more complex hierarchy of structures and the possibility that the paths to the fields will change in the future.

For example, this is the structure:

type Second struct {
    Items []string
}

type First struct {
    Store map[string]Second
}

We won't be able to write like this:

a := First{}

a.Store["nike"].Items = append(a.Store["nike"].Items, "Air")

An error will be received:

cannot assign to struct field a.Store["nike"].Items in map

How about this? 🤔

a := First{}

a.Store["nike"] = Second{
    Items: append(a.Store["nike"].Items, "Air"),
}

Ah yes, you have to initiate the map first...

panic: assignment to entry in nil map

If you need a quick and short solution, this would be it:

func main() {
    a := First{
        Store: make(map[string]Second),
    }

    a.Store["nike"] = Second{
        Items: append(a.Store["nike"].Items, "Air"),
    }
}

But, more correct with all the checks, it should have been written like this:

func main() {
    a := First{
        Store: make(map[string]Second), // Initialize the map
    }

    key := "nike"
    value := "Air"

    // Check if the key exists in the map
    if second, ok := a.Store[key]; ok {
        // Append the new value to the existing Second struct
        second.Items = append(second.Items, value)
        a.Store[key] = second
    } else {
        // Create a new Second struct with the new value and add it to the map
        a.Store[key] = Second{
            Items: []string{value},
        }
    }

    fmt.Println(a.Store["nike"].Items)
}

Quite a trivial process if you have a simple structure, but if you have several nested structures with data? This often happens when working with JSON data and their conversion as a structure.

Applying the package: Dot 🔥

Using the dot package, all the work would be reduced to this code:

func main() {
    a := First{}

    // Passing the structure to Dot
    obj, _ := dot.New(&a)

    // Specify the path to the field from the structure and the value
    err := obj.Insert("Store.nike.Items.-1", "Air")
}

Description of the path:

Store is a field from the First structure
nike is the map key;
Items is a field from the Second structure;
-1 that's probably the weirdest thing. It means it's a slice and a value needs to be added at the very end or a new one needs to be initialised.

Features

Dot-separated Path Insertion: Insert values into struct fields, arrays, slices, maps, and channels using a dot-separated path like Field1.Field2.Field3.
Nested Struct Support: Capable of handling complex, deeply nested Go structs.
Flexible Value Type: Accepts any Go value types, allowing for wide-ranging manipulations.
Support for Collection Types: This package not only works with struct fields but also supports insertion into slices, arrays, maps, and even channels, making it a truly versatile tool for manipulating data structures in Go.
Map Key Placeholders: Enables the use of placeholders for map keys, providing a flexible way to interact with Go maps.
Array and Slice Indexing: Replace values in an array or a slice by specifying an index in the path. This feature makes it easy to modify specific elements without having to iterate over the entire data structure.
Slice Appending: Easily add a new value at the end of a slice by specifying -1 in the path. This intuitive feature simplifies dynamic data modification, enhancing the versatility of your Go programs.
Smart Value Replacing: Whether you're working with basic types or complex data structures, dot offers intelligent replacing that keeps your data intact and your code clean.

Whether you are aiming to replace a specific element in a slice or add a new entry to a map, dot gets you there effortlessly. Backed by smart placeholder handling, the package ensures smooth operation with map keys of varied types, giving you the power to modify data structures precisely the way you want.

Getting Started

Getting started with dot is straightforward. To use dot in your Go application, you will first need to download and install it. This can be done with the go get command, as shown below:

go get github.com/mowshon/dot

After installing the package, you can import it into your Go files with the following line of code:

import "github.com/mowshon/dot"

You are now ready to use dot to effortlessly manipulate complex data structures in Go. The upcoming sections of this documentation will guide you through the various methods and features of dot, each complete with illustrative examples and detailed explanations.

Stay tuned to discover the power and elegance of dot, your tool for precise and convenient data manipulations in Go!

Constructor Usage

One of the first steps when working with dot is creating an instance using your target structure. This can be done using dot's constructor method, New.

Passing a Variable to the Constructor

dot.New is the constructor method that creates and returns a new dot instance. The method requires one argument: a pointer to the variable that you want to manipulate. This variable is typically a struct, but it can also be a slice, array, map, or channel.

Here is a basic example of how to create a dot instance:

type MyStruct struct {
    Field1 struct {
        Field2 string
    }
}

func main() {
    data := MyStruct{}
    obj, err := dot.New(&data)

    if err != nil {
        // handle error
    }

    // You can now use obj to manipulate data...
}

In the example above, we first define a struct named MyStruct that we want to manipulate. Inside the main function, we create an instance of MyStruct, data. We then pass a pointer to data to dot.New, which returns a dot instance (obj) and an error (err).

If dot.New encounters an error while creating the dot instance, it will return a non-nil err. Always check err to ensure that the dot instance was created successfully.

After successful creation, you can use the dot instance (obj in the example) to perform various operations on data, which will be detailed in the following sections.

With the dot instance in your hands, you're ready to dive into the functionalities provided by dot. The next sections will explore the Insert method and its versatility in modifying different data types and structures in Go.

Insertion into Structure Fields

Once you've instantiated a dot object using the constructor, you can start leveraging its capabilities to manipulate data structures. One of the most powerful features dot offers is the Insert method. It allows you to insert values into struct fields using a dot-separated path.

Insert Method Syntax

The Insert method requires two arguments:

The first argument is a string that represents the path to the field where you want to insert the value. The path should be constructed as a dot-separated sequence of field names starting from the topmost struct field name.
The second argument is the value you want to insert. This can be any Go data type.

Here's an example of how you can insert a value into a struct field using the Insert method:

type MyStruct struct {
    Field1 struct {
        Field2 string
    }
}

func main() {
    data := MyStruct{}
    obj, err := dot.New(&data)

    if err != nil {
        // handle error
    }

    err = obj.Insert("Field1.Field2", "My Value")

    if err != nil {
        // handle error
    }

    fmt.Println(data.Field1.Field2) // Prints: My Value
}

In this example, we're inserting the string "My Value" into the field Field2 which is nested inside Field1 of our struct MyStruct. The Insert method traverses the dot-separated path "Field1.Field2" to reach the desired location and then inserts the provided value.

If the Insert method encounters an error while trying to insert the value, it will return a non-nil err. Always check err to ensure that the operation was successful.

With dot, even the most deeply nested struct fields are just a dot-separated path away! The upcoming sections will provide insights on how to work with more complex data structures using dot. Stay tuned!

Working with Maps

dot not only allows you to handle regular structure fields but also supports complex data types like maps. This section will guide you on how to use the Insert method to manipulate maps in your structures.

Inserting a Value into a Map Field

A map is a built-in data type in Go that associates values with keys. To insert a value into a map field, you can use the Insert method in the same way as for struct fields. The only difference is that the path will now include a map key.

Here's an example:

type MyStruct struct {
    MyMap map[string]int
}

func main() {
    data := MyStruct{}
    obj, err := dot.New(&data)

    if err != nil {
        // handle error
    }

    err = obj.Insert("MyMap.year", 2023)

    if err != nil {
        // handle error
    }

    fmt.Println(data.MyMap["year"]) // Prints: 2023
}

In this example, we're inserting the integer 2023 into the map MyMap with the key "year". The Insert method finds the map using the first part of the path ("MyMap") and then inserts the value at the map key specified in the path.

Again, don't forget to check the err returned by the Insert method to ensure that the operation was successful.

Being able to manipulate maps so conveniently brings you a step closer to mastering the management of complex data structures with dot. Next, we'll explore how to use dot to work with slices and arrays.

Insertion and Replacement in Slices

dot is a versatile tool not only for handling struct fields and maps but also for working with dynamic collections such as slices. This section will explain how you can use the Insert method for modifying slices in your structures.

Inserting a Value at the End of a Slice

To insert a new value at the end of a slice, you can use the Insert method and specify -1 in the path. Here's how it works:

type Data struct {
    Title string
}

type MyStruct struct {
    Field3 []Data
}

func main() {
    data := MyStruct{}
    obj, err := dot.New(&data)

    if err != nil {
        // handle error
    }

    err = obj.Insert("Field3.-1", Data{Title: "new Title"})

    if err != nil {
        // handle error
    }

    fmt.Println(data.Field3[0].Title) // Prints: new Title
}

In the above example, Data{Title: "new Title"} is added to the end of the slice Field3. The -1 in the path "Field3.-1" indicates that the new value should be appended to the slice.

Replacing a Value by Slice Index

To replace a value at a specific index in a slice, you can use the Insert method and specify the index in the path. Here's an example:

type Data struct {
    Title string
}

type MyStruct struct {
    Field3 []Data
}

func main() {
    data := MyStruct{Field3: []Data{{Title: "old Title"}}}
    obj, err := dot.New(&data)

    if err != nil {
        // handle error
    }

    err = obj.Insert("Field3.0.Title", "replace Title")

    if err != nil {
        // handle error
    }

    fmt.Println(data.Field3[0].Title) // Prints: replace Title
}

In this example, the string "replace Title" replaces the string at the specified index in the slice Field3. The index 0 in the path "Field3.0.Title" determines the position of the slice element that will be modified.

With dot, it's easy to modify slices in your structures. Next, we'll see how to use dot to work with arrays.

Working with Arrays

Arrays in Go are fixed-length sequences of items of the same type. dot provides the functionality to modify array elements using its Insert method. However, as arrays have a fixed length, you cannot append new elements to them, so the -1 value in the path is not applicable.

Here's an example of how to replace a value in an array using dot:

type MyStruct struct {
    Field5 [3]int
}

func main() {
    data := MyStruct{}
    obj, _ := dot.New(&data)

    err := obj.Insert("Field5.1", 2023)

    if err != nil {
        // handle error
    }

    fmt.Println(data.Field5[1]) // Prints: 2023
}

In this example, Field5 is an array of integers of size 3. The Insert method replaces the integer at index 1 with 2023. The index 1 in the path "Field5.1" determines the position of the array element that will be modified.

Using dot, it's straightforward to modify arrays in your structures, giving you an extra tool in your belt for handling complex data structures in Go.

Insertion into Channels

Channels are a powerful feature in Go that provide a way for two goroutines to communicate with each other and synchronize their execution. dot gives you the ability to insert values into channels with its Insert method.

Inserting a Value into a Channel

The Insert method can be used to send a value into a channel field in your structures. The following example demonstrates how to do this:

type MyStruct struct {
    FieldChannel chan string
}

func main() {
    data := MyStruct{}
    obj, err := dot.New(&data)

    if err != nil {
        // handle error
    }

    go func() {
        err = obj.Insert("FieldChannel", "value for channel")

        if err != nil {
            // handle error
        }
    }()

    message := <-data.FieldChannel
    fmt.Println(message) // Prints: value for channel
}

In the above example, "value for channel" is sent to the FieldChannel channel. The Insert method takes care of the necessary operations to send the value into the channel. This operation is equivalent to the Go code:

data.FieldChannel <- "value for channel"

This concludes the basic usage of the Insert method in different contexts. Next, we'll discuss an advanced feature of dot that allows you to deal with map keys of different types using placeholders.

Working with Different Map Keys: Placeholders

Working with maps where keys are not strings can be tricky. But don't worry, dot has you covered. dot provides the ability to define placeholders for map keys of different types.

Defining and Replacing Placeholders

A placeholder in dot is a representation of a map key of any type. To define a placeholder, you can use the Replace method.

Here's how you can define and use placeholders in dot:

type Key string
const UniqueID Key = "Some"

type MyStruct struct {
    Field4 map[Key]string
}

func main() {
    data := MyStruct{}
    obj, _ := dot.New(&data)

    // Define placeholder
    obj.Replace("First", UniqueID)

    // Use placeholder in path
    err := obj.Insert("Field4.First", "value for map")

    if err != nil {
        // handle error
    }

    fmt.Println(data.Field4[UniqueID]) // Prints: value for map
}

In this example, First is defined as a placeholder for UniqueID using the Replace method. Once the placeholder is defined, you can use it in the path string passed to the Insert method.

When the Insert method encounters a placeholder in the path, it replaces the placeholder with its corresponding map key before inserting the value. This operation is equivalent to the Go code:

data.Field4[UniqueID] = "value for map"

This concludes our discussion on placeholders in dot, rounding out your understanding of dot's powerful features for dealing with complex data structures.

Pros, Cons and Use Cases

While the dot package provides great flexibility and convenience when working with complex data structures in Go, there are some considerations and potential disadvantages to keep in mind:

Performance Impact: The use of reflection, which dot relies on to access and manipulate data structures, can be slower compared to direct field access and manipulation. This might not be an issue for smaller applications, but for high-performance applications where every millisecond counts, this could be a potential drawback.
Error Handling: The package returns an error if the path is not found or if there is a type mismatch. While this is good for catching mistakes, it does mean that you need to handle these errors properly, which could add additional complexity to your code.
Loss of Type Safety: One of the benefits of Go is its strong type system which can catch many errors at compile time. However, with the dot package, since you're specifying paths as strings and using reflection, some errors can only be caught at runtime, which may lead to increased debugging time.

Despite these potential drawbacks, the dot package can be incredibly useful in various situations:

Data Transformations: If you frequently need to transform complex data structures (e.g., nested structs, maps, arrays), the dot package can simplify this process significantly.
Configuration Management: If your Go application works with complex configuration data stored in nested structures, dot can make it easier to retrieve and update configuration values.
JSON Path-like Operations: If you need to perform JSONPath-like operations but on Go data structures, dot provides a similar functionality.

As always, the decision to use a package like dot should be based on your specific use case, taking into account the trade-offs between readability, maintainability, performance, and development speed.

Benchmark Results

We conducted a benchmark comparison between the dot package and the native Golang style for data insertion into different data types in a 5-level nested structure. Here are the results:

BenchmarkDotInsert-12             320488          3536 ns/op
BenchmarkNativeInsert-12        442920729            2.661 ns/op

Interpretation

BenchmarkDotInsert-12 is the function that tests the performance of our dot package. The benchmark was able to perform 320,488 iterations of the function in the default time (1 second), with each operation taking 3,536 nanoseconds (ns) or 3.536 microseconds (µs).
BenchmarkNativeInsert-12 is the function that tests the performance of the equivalent native Golang operations. The benchmark was able to perform a staggering 442,920,729 iterations in the default time, with each operation taking only 2.661 nanoseconds (ns).

Conclusion

Based on the benchmark results, it's evident that the native Golang operations are faster than the operations performed by the dot package. However, speed isn't everything when it comes to code. While the native Golang style may be faster, it can also be more verbose and complex, especially when dealing with deeply nested data structures.

DEV Community: St. B

Iterium - Generic Channel-based Iterators for Golang

Contents

Installation

Import

Decrypting the MD5 hash in Golang

Benchmark ⏰

🔑 How many total combinations of possible passwords did it take to crack a 6-character md5 hash?

Iterator architecture

Creating an Iterator

Getting data from an iterator

Combinatoric iterators

🟢 iterium.Product([]T, length) - Cartesian Product

🟢 iterium.Permutations([]T, length)

🟢 iterium.Combinations([]T, length)

🟢 iterium.CombinationsWithReplacement([]T, length)

Infinite iterators

🔴 iterium.Count(start, step)

🔴 iterium.Cycle(Iterator)

🔴 iterium.Repeat(value, n)

Finite iterators

🔵 iterium.Range(start, stop, step)

Features 🔥

🔵 iterium.Map(iter, func)

Calculating the Fibonacci Number with Iterium

🔵 iterium.StarMap(iter, func)

🔵 iterium.Filter(iter, func)

🔵 iterium.FilterFalse(iter, func)

🔵 iterium.Accumulate(iter, func)

🔵 iterium.TakeWhile(iter, func)

🔵 iterium.DropWhile(iter, func)

Create your own iterator 🛠️

Go: Insert a value into nested structures with a dot

Connect.the.Dots, Control.your.Data!

Applying the package: Dot 🔥

Features

Getting Started

Constructor Usage

Passing a Variable to the Constructor

Insertion into Structure Fields

Insert Method Syntax

Working with Maps

Inserting a Value into a Map Field

Insertion and Replacement in Slices

Inserting a Value at the End of a Slice

Replacing a Value by Slice Index

Working with Arrays

Insertion into Channels

Inserting a Value into a Channel

Working with Different Map Keys: Placeholders

Defining and Replacing Placeholders

Pros, Cons and Use Cases

Benchmark Results

Interpretation

Conclusion