DEV Community: Kirill Scherba

Ordered map

Kirill Scherba — Sat, 15 Feb 2025 11:51:43 +0000

Omap is Golang package for working with thread safe ordered maps. The ordered map contains the golang map, list and mutex to execute Ordered Map functions.

The Ordered Map is a map that remembers the order of items. The map can be iterated over to retrieve the items in the order they were added.

Introduction to the omap Go Package

The omap Go package is a lightweight and efficient library for working with ordered maps in Go. An ordered map is a data structure that combines the benefits of a map and a list, allowing you to store key-value pairs in a specific order.

What is omap?

Omap is a Go package that provides an implementation of an ordered map. It is designed to be fast, efficient, and easy to use. Omap is particularly useful when you need to store data in a specific order, such as when working with configuration files, caching, or data processing pipelines.

Key Features of omap

Ordered: omap preserves the order in which key-value pairs are inserted, allowing you to iterate over the map in a specific order.
Fast lookups: omap uses a hash table to store key-value pairs, making lookups fast and efficient.
Efficient insertion and deletion: omap uses a linked list to store the order of key-value pairs, making insertion and deletion operations efficient.

Using omap

To use omap, you can install it using the following command:

go get github.com/kirill-scherba/omap

Here is an example of how to use omap:

package main

import (
    "fmt"
    "log"

    "github.com/kirill-scherba/omap"
)

func main() {
    // Create a new ordered map
    m, err := omap.New[string, string]()
    if err != nil {
        log.Fatal(err)
    }

    // Insert some key-value pairs
    m.Set("key1", "value1")
    m.Set("key2", "value2")
    m.Set("key3", "value3")

    // Iterate over the omap in order
    for _, pair := range m.Pairs() {
        fmt.Printf("%s: %s\n", pair.Key, pair.Value)
    }
}

This code creates a new omap, inserts some key-value pairs, and then iterates over the omap in order, printing out each key-value pair.

Conclusion

The omap Go package is a useful library for working with ordered maps in Go. Its fast lookups, efficient insertion and deletion, and ordered iteration make it a great choice for a variety of use cases. Whether you're working with configuration files, caching, or data processing pipelines, omap is definitely worth considering.

Example Use Cases

Configuration files: Use omap to store configuration data in a specific order, making it easy to iterate over the configuration and apply settings in the correct order.
Caching: Use omap to store cached data in a specific order, making it easy to iterate over the cache and evict items in the correct order.
Data processing pipelines: Use omap to store data in a specific order, making it easy to iterate over the data and process it in the correct order.

See more description and examples on omap projects github page:
https://github.com/kirill-scherba/omap/

Teonet messages queue

Kirill Scherba — Thu, 21 Mar 2024 07:29:45 +0000

The Teonet messages queue is a part of the Teonet network.

A message queue is a form of asynchronous service-to-service communication used in serverless and microservices architectures. Messages are stored on the queue until they are processed and deleted. Each message is processed only once, by a single consumer. Message queues can be used to decouple heavyweight processing, to buffer or batch work, and to smooth spiky workloads.

Basic teomq scheme

The Teonet messages queue contains three basic parts:

Teonet messages broker
Teonet messages consumer
Teonet messages producer

See more at: https://github.com/teonet-go/teomq

Teonet messages queue

Kirill Scherba — Thu, 21 Mar 2024 07:29:41 +0000

The Teonet messages queue is a part of the Teonet network.

Basic teomq scheme

The Teonet messages queue contains three basic parts:

Teonet messages broker
Teonet messages consumer
Teonet messages producer

See more at: https://github.com/teonet-go/teomq

Golang multinode tree with parallel search

Kirill Scherba — Wed, 06 Sep 2023 09:10:26 +0000

Golang package tree is pure golang implementation of tree with any direction nodes. Each node in the tree can be connected to many children and has many parents.

In the implementation of this tree, parents are no different from children. The tree element contains references to branches without specifying whether it is a parent or a child.

Each node reference in an element contains a path cost that is taken into account when calculating the path from point A to point B. The path between points A and B calculates using threads (parallel) to minimize time of find all paths.

This tree has no root. Paths in a tree can be looped.

The tree elements can store any data structure. This structure is defined when the tree is created.

See the part of code to create Tree and it Elements:



t := tree.New[TreeData]("My first element")

// Create new tree elements (first element and end point element)
e := t.New("My first element")
ep := t.New("End point")

// Create children of e (first element) element
ch1, _ := e.Add(t.New("My first child"))
ch2, _ := e.Add(t.New("My second child"), tree.WayOptions{Cost: 3.0})
ch4, _ := e.Add(t.New("My fourth child"), tree.WayOptions{Cost: 3.0})

// Create sub children elements
ch3, _ := ch2.Add(t.New("Some third child"))
ch3.Add(ch4)

// Add children to ep (end point) element
ch1.Add(ch4)
ch2.Add(ep)
ch4.Add(ep)

// Set oneway path from ep (end point) to e (first element) element
ep.Add(e, tree.WayOptions{Cost: 5.0, Oneway: true})

// Print tree started from e (first element) element
fmt.Printf("\nPrint tree:\n%v\n\n", e)

The result of last printf:



. My first element
├── My first child, cost: 1.00
│   ├── My fourth child, cost: 1.00
│   │   ├── My first element, cost: 3.00 🡡
│   │   ├── Some third child, cost: 1.00
│   │   │   ├── My second child, cost: 1.00
│   │   │   │   ├── My first element, cost: 3.00 🡡
│   │   │   │   ├── Some third child, cost: 1.00 🡡
│   │   │   │   └── End point, cost: 1.00
│   │   │   │       ├── My second child, cost: 1.00 🡡
│   │   │   │       ├── My fourth child, cost: 1.00 🡡
│   │   │   │       └── My first element, cost: 5.00 (one way road) 🡡
│   │   │   └── My fourth child, cost: 1.00 🡡
│   │   ├── My first child, cost: 1.00 🡡
│   │   └── End point, cost: 1.00 🡡
│   └── My first element, cost: 1.00 🡡
├── My second child, cost: 3.00 🡡
├── My fourth child, cost: 3.00 🡡
└── End point, cost: 5.00 (way not allowed) 🡡

See the code and more descriptions and example at: https://github.com/kirill-scherba/tree

Connect to Tru peers using unix socket

Kirill Scherba — Tue, 22 Aug 2023 12:01:38 +0000

If you can't link the Tru package to your application than use this standalone unix socket server to communicate with any Tru peers.

The Unix socket Server created on golang and can run under Linux or Windows.

The Unix socket Cliens examples created on Go, C, C++ and build-in sample Visual Studio project.

See the part of C++ code to connect to the Unix socket Server:

int main(int argc, char *argv[]) {

  // Get unix socket path
  std::string socket_path = std::string("/tmp/trugw.sock");

  // Get tru peer address
  std::string tru_addr = ":7070";
  if (argc >= 2) {
    tru_addr = std::string(argv[1]);
  }

  std::cout << "Trugw C++ client, "
            << "sock path: " << socket_path << "tru peer: " << socket_path
            << std::endl;

  // Connect to teogw server
  std::cout << "trying to connect...\n";
  Trugw tgw(socket_path, tru_addr);
  if (!tgw.connected()) {
    std::cout << "can't connect\n";
    return 1;
  }
  std::cout << "connected \n";

  // Send messages
  for (int i = 0; i < 50000; i++) {
    std::string msg = "Hello " + std::to_string(i);
    std::cout << "send " << msg << std::endl;
    tgw.send(msg);

    uint8_t buf[1024];
    auto n = tgw.recv((const char *)buf, sizeof(buf), 0);
    std::string s((const char *)buf, n);
    std::cout << "receive " << s << std::endl;
  }

  return 0;
}

See the code and more descriptions and example at: https://github.com/teonet-go/trugw

Tru go net example

Kirill Scherba — Sun, 30 Apr 2023 11:35:17 +0000

New version of Tru (Teonet reliable UDP) golang package was releaset.

The Tru golang net module and example was added. This example is Client/Server application which transfer data using golang net standard functions.

https://github.com/teonet-go/tru

TeoS3 package and s3cp utility

Kirill Scherba — Fri, 04 Nov 2022 21:34:51 +0000

The TeoS3 package contains Golang features that make it easy to use S3 storage as a key-value database.

This project contain also the s3cp utility which copy files from disk to s3 storage and back.

https://github.com/teonet-go/teos3

There is a part of code with connect, set and get key value:

// Connect to S3 storage
con, err := teos3.Connect(accessKey, secretKey, endpoint, secure)
if err != nil {
    log.Fatalln(err)
}

// Set key to teos3 Map
err = con.Map.Set(key, data)
if err != nil {
    log.Fatalln(err)
}

// Get key from TeoS3 Map
data, err := con.Map.Get(key)
if err != nil {
    log.Fatalln(err)
}

See the code and more descriptions and example at: https://github.com/teonet-go/teos3

Teonet C library

Kirill Scherba — Wed, 08 Jun 2022 15:47:49 +0000

Teonet news.

Today we published Teonet v5 C library and examples project.

Teonet is designed to create client-server systems and build networks for server applications operating within a microservice architecture.

Main Teonet v5 code was created on Golang. This project make c/c++ dynamic library to use it in any C/C++ and C compatible applications.

The description of Teonet you can find at main Teonet-go page.

The examples of this project copy Teonet-go examples functional. So look this project examples to get started with teonet-c library: examples

See the project code and description on Github:
https://github.com/teonet-go/teonet-c

We are waiting your support! Please give stars to our Teonet-c and Teonet projects in Github.

Best regards,
Kirill Scherba
https://github.com/kirill-scherba

Tunnel between hosts without public IPs

Kirill Scherba — Sun, 05 Jun 2022 17:26:28 +0000

There is Teonet news.

The Teotun packet and application was published today by Teonet.

The Teotun creates secret tunnel between hosts without public IPs using Teonet. The connection based on TRU transport and create reliable, low latency, encrypted P2P channels between connected peers.

See the project code and description on Github:

https://github.com/teonet-go/teotun

We are waiting your support! Please give stars to our Teotun and Teonet projects in Github.

Best regards,
Kirill Scherba
https://github.com/kirill-scherba

Create secret tunnel using Go

Kirill Scherba — Mon, 30 May 2022 13:25:14 +0000

Trutun

Look at my new application Trutun.

It creates secret tunnel by IP address using Teonet TRU transport. TRU create reliable, low latency and encrypted channel between connected peers.

Star

The repository is available on Github:
https://github.com/teonet-go/trutun

Using a neural network in Go

Kirill Scherba — Wed, 25 May 2022 20:37:47 +0000

NNHelper is a Go package for creating and using a neural network

Summary

This article describes the work of the nnhelper package, designed to create and use neural networks in Go programs.

If you are already familiar with machine learning and use it in your work, then this article and the examples described in it may seem too simple for you. If you are at the beginning of your journey and want to get acquainted with this topic or would like to learn how to use the neural network in your Go programs, then you have come to the right place.

The nnhelper Go package is designed to quickly create a neural network and use it in the applications written in the Go language. To use nnhelper, you don't need anything else other than Go. The nnhelper package is an add-on to the gonn package. And this is the only one external dependency.

Neural network (neural matrix)

Let me try to explain in my own words what a neural network or neural matrix is, in order not to copy here complex and, perhaps, incomprehensible explanations from the initial reading.

A neural matrix is an array of floating point values that allows you to give the desired output values depending on the input values. The values known to us are fed to the input of the neural matrix, and the results we expect are given at the output.

Let's use the neural matrix to solve the problem "Two Minuses Make a Plus." To do this, we will make two tables: one with the input data, the other with the results.

We have two input parameters and two output parameters. The input is two numbers with a plus or minus sign. At the output we get two numbers with values from 0 to 1. If the first value is close to 1, then the result is Plus, if the second value is close to 1, then the result is Minus. That is, we give real values as input, and we get an array of results as output. Than we select the largest value from the array and consider it the answer.

Input data:              Results(plus, minus):
 1, 1  – plus * plus     1, 0 – plus
 1,-1  – plus * minus    0, 1 – minus
-1, 1  – minus* plus     0, 1 – minus
-1,-1  – minus* minus    1, 0 – plus

From the prepared tables with input and output data, a neural matrix is created by its training: The input data is fed to the inputs of the matrix. The matrix produces an output that is compared with the result from our results table. Based on these answers, the coefficients in the matrix change. The input data is looped through hundreds, thousands and even hundreds of thousands of iterations until the matrix answers are accurate enough.

Let's move on to programming

Enough words and theory. Let's see how it looks in practice.

Let's create a folder for the project. And create three files in it:

main.go 
sam03_inp.csv 
sam03_tar.csv

Let's place our input and output data in the *.csv files:

sam03_inp.csv

1,1
1,-1
-1,1
-1,-1

sam03_tar.csv

1,0
0,1
0,1
1,0

In the main.go file, create the main function and write the following code:

// Constants with filenames of our matrix and data
const (
    SAM03_NN  = "sam03.nn"
    SAM03_INP = "sam03_inp.csv"
    SAM03_TAR = "sam03_tar.csv"
)

// Human answers string array
humanAnswers := []string{"Plus", "Minus"}

// Create NN if it file does not exists
if _, err := os.Stat(SAM03_NN); errors.Is(err, os.ErrNotExist) {
        log.Println("Create", SAM03_NN, "neural network")
        nnhelper.Create(2, 4, 2, false, SAM03_INP, SAM03_TAR, SAM03_NN, true)
}

// Load neural matrix from file
nn := nnhelper.Load(SAM03_NN)

// Using / testing our neural network
const (
        PLUS  = 1.0
        MINUS = -1.0
    )

    // Intput array for testing
    in := [][]float64{
        {PLUS, PLUS},   // Plus * Plus = Plus
        {PLUS, MINUS},  // Plus * Minus = Minus
        {MINUS, PLUS},  // Minus * Plus = Minus
        {MINUS, MINUS}, // Minus * Minus = Plus
        {3000, -0.001}, // Minus * Plus = Minus
    }
    for i := range in {
        out := nn.Answer(in[i]...)
        answer, _ := nn.AnswerToHuman(out, humanAnswers)
        fmt.Println(in[i], answer, out)
    }
}

The full text of this example and the data files are located in examples/sam03

Let's run the example:

go run .

And we get the results:

[1 1] Plus [0.9944239772210877 0.005449692189449571]
[1 -1] Minus [0.006860785779850435 0.9935960167863507]
[-1 1] Minus [0.005651009980489101 0.994384581174021]
[-1 -1] Plus [0.9944591181959666 0.005221796400203198]
[3000 -0.001] Minus [0.005445102841471242 0.9960123783099599]

In the results we see (see the first line):

our original data: [1,1]
result translated into understandable form: Plus
result obtained from matrix outputs: [0.9944239772210877 0.005449692189449571]

Pay attention to the last line of the results. Our matrix was able to give the correct answer to the "unknown" inputs. This is the whole taste of neural networks. The matrix gives answers not only to the input data that it was trained on, but also to other input parameters unknown to it.

To see the training process of the neural network, you need to delete the sam03.nn file and run the example again.

Well, I guess I'll wrap it up here. For a long time I dreamed of writing a clear and simple explanation of how neural networks can be used in programming. I hope I succeeded.

There are two more examples in the package:

sam02: Input the time in 24-hour format and get the answer: Morning, Evening, Day or Night;
sam01: An example of a matrix for getting the reaction of a game bot. The input is the amount of health, the presence of weapons, the number of enemies, and at the output we get the answer to the question “what to do”: attack, sneak, run away or do nothing.

The package is hosted on Github:
https://github.com/kirill-scherba/nnhelper

Best regards,
Kirill Scherba