OpenSearch CRUD operation in Go

Introduction:

OpenSearch, an open-source search and analytics engine, provides a robust platform for storing, indexing, and searching data. As a Go developer, you can leverage the OpenSearch Go SDK to perform CRUD (Create, Read, Update, Delete) operations programmatically. In this article, we will explore how to perform CRUD operations in OpenSearch using Go and the OpenSearch Go SDK.

In my previous example I have explained the CURL operations via curl.
View Article

Prerequisites:

To follow along with the examples in this article, you'll need to have Go installed on your system and have access to an OpenSearch cluster.

Run the open-search cluster in docker-compose file

creating docker-compose.yml file

version: '3'
services:
  opensearch-node1: # This is also the hostname of the container within the Docker network (i.e. https://opensearch-node1/)
    image: opensearchproject/opensearch:latest # Specifying the latest available image - modify if you want a specific version
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster # Name the cluster
      - node.name=opensearch-node1 # Name the node that will run in this container
      - discovery.seed_hosts=opensearch-node1,opensearch-node2 # Nodes to look for when discovering the cluster
      # - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 # Nodes eligible to serve as cluster manager
      - bootstrap.memory_lock=true # Disable JVM heap memory swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # Set min and max JVM heap sizes to at least 50% of system RAM
    ulimits:
      memlock:
        soft: -1 # Set memlock to unlimited (no soft or hard limit)
        hard: -1
      nofile:
        soft: 65536 # Maximum number of open files for the opensearch user - set to at least 65536
        hard: 65536
    volumes:
      - opensearch-data1:/usr/share/opensearch/data # Creates volume called opensearch-data1 and mounts it to the container
    ports:
      - 9200:9200 # REST API
      - 9600:9600 # Performance Analyzer
    networks:
      - opensearch-net # All of the containers will join the same Docker bridge network

  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:latest # Make sure the version of opensearch-dashboards matches the version of opensearch installed on other nodes
    container_name: opensearch-dashboards
    ports:
      - 5601:5601 # Map host port 5601 to container port 5601
    expose:
      - "5601" # Expose port 5601 for web access to OpenSearch Dashboards
    environment:
      OPENSEARCH_HOSTS: '["https://opensearch-node1:9200"]' # Define the OpenSearch nodes that OpenSearch Dashboards will query
    networks:
      - opensearch-net

volumes:
  opensearch-data1:
  opensearch-data2:

networks:
  opensearch-net:

To verify this cluster is running:

• run command in terminal to

curl https://localhost:9200/_cat/indices -ku 'admin:admin'

• open the url for dashboard http://0.0.0.0:5601/ access it with username:admin and password: admin

To run the command in dashboard

Setting up the OpenSearch Go SDK:

Before we dive into the CRUD operations, we need to set up the OpenSearch Go SDK. The official OpenSearch Go SDK, also known as opensearch-go, provides a simple and intuitive way to interact with OpenSearch using Go. To install the SDK, use the following command:

go get github.com/opensearch-project/opensearch-go/v2

Performing CRUD Operations:

Now that we have the OpenSearch Go SDK set up, let's dive into the CRUD operations:
You can also refer the code in repo here: https://github.com/ankitmalikg2/opensearch-crud

Creating an Index:

To create an index in OpenSearch, we need to initialize a client and use the CreateIndex API. Here's an example:

package main

import (
    "crypto/tls"
    "fmt"
    "net/http"

    opensearch "github.com/opensearch-project/opensearch-go/v2"
)

func main() {
    // Replace with your OpenSearch cluster details
    endpoint := "https://localhost:9200"
    username := "admin" // Leave empty if not using authentication
    password := "admin" // Leave empty if not using authentication

    // Create a client
    client, err := opensearch.NewClient(opensearch.Config{
        Addresses: []string{endpoint},
        Username:  username,
        Password:  password,
        Transport: &http.Transport{
            TLSClientConfig: &tls.Config{
                InsecureSkipVerify: true,
            },
        },
    })
    if err != nil {
        fmt.Println("Error creating OpenSearch client:", err)
        return
    }

    // Create an index
    indexName := "dev-article"
    err = createIndex(client, indexName)
    if err != nil {
        fmt.Println("Error creating index:", err)
        return
    }
    fmt.Println("Index created:", indexName)
}

func createIndex(client *opensearch.Client, indexName string) error {

    _, err := client.Indices.Create(indexName)

    if err != nil {
        return err
    }

    return nil
}

output:

Index created: dev-article

Indexing or Adding a Document:

To index a document in OpenSearch, we can use the Index API. Here's an example:

package main

import (
    "crypto/tls"
    "fmt"
    "net/http"

    "github.com/opensearch-project/opensearch-go/opensearchutil"
    opensearch "github.com/opensearch-project/opensearch-go/v2"
)

func main() {
    // Replace with your OpenSearch cluster details
    endpoint := "https://localhost:9200"
    username := "admin" // Leave empty if not using authentication
    password := "admin" // Leave empty if not using authentication

    // Create a client
    client, err := opensearch.NewClient(opensearch.Config{
        Addresses: []string{endpoint},
        Username:  username,
        Password:  password,
        Transport: &http.Transport{
            TLSClientConfig: &tls.Config{
                InsecureSkipVerify: true,
            },
        },
    })
    if err != nil {
        fmt.Println("Error creating OpenSearch client:", err)
        return
    }

    // Index a document
    indexName := "dev-article"
    documentID := "1"
    document := map[string]interface{}{
        "title":   "Getting Started with OpenSearch",
        "content": "OpenSearch is a powerful open-source search and analytics engine...",
    }
    err = indexDocument(client, indexName, documentID, document)
    if err != nil {
        fmt.Println("Error indexing document:", err)
        return
    }
    fmt.Println("Document indexed:", documentID)
}

func indexDocument(client *opensearch.Client, indexName string, documentID string, document map[string]interface{}) error {

    _, err := client.Create(indexName, documentID, opensearchutil.NewJSONReader(document))

    return err
}

Output:

Document indexed: 1

Retrieving a Document:

To retrieve a document from OpenSearch, we can use the Get API. Here's an example:

package main

import (
    "context"
    "fmt"
    opensearch "github.com/opensearch-project/opensearch-go"
    "github.com/opensearch-project/opensearch-go/opensearchapi"
)

func main() {
    // Replace with your OpenSearch cluster details
    endpoint := "http://localhost:9200"
    username := "" // Leave empty if not using authentication
    password := "" // Leave empty if not using authentication

    // Create a client
    client, err := opensearch.NewClient(opensearch.Config{
        Addresses: []string{endpoint},
        Username:  username,
        Password:  password,
    })
    if err != nil {
        fmt.Println("Error creating OpenSearch client:", err)
        return
    }

    // Retrieve a document
    indexName := "dev-article"
    documentID := "1"
    retrievedDocument, err := getDocument(client, indexName, documentID)
    if err != nil {
        fmt.Println("Error retrieving document:", err)
        return
    }
    fmt.Println("Retrieved Document:", retrievedDocument)
}

func getDocument(client *opensearch.Client, indexName string, documentID string) (map[string]interface{}, error) {
    getRequest := opensearchapi.GetRequest{
        Index: indexName,
        ID:    documentID,
    }

    response, err := client.Get(&getRequest)
    if err != nil {
        return nil, err
    }

    document := response.Source

    return document, nil
}

Output:

Retrieved Document: map[_id:1 _index:dev-article _primary_term:1 _seq_no:0 _source:map[content:OpenSearch is a powerful open-source search and analytics engine... title:Getting Started with OpenSearch] _version:1 found:true]

Updating a Document:

To update a document in OpenSearch, we can use the Update API. Here's an example:

package main

import (
    "crypto/tls"
    "fmt"
    "net/http"

    opensearch "github.com/opensearch-project/opensearch-go/v2"
    "github.com/opensearch-project/opensearch-go/v2/opensearchutil"
)

func main() {
    // Replace with your OpenSearch cluster details
    endpoint := "https://localhost:9200"
    username := "admin" // Leave empty if not using authentication
    password := "admin" // Leave empty if not using authentication

    // Create a client
    client, err := opensearch.NewClient(opensearch.Config{
        Addresses: []string{endpoint},
        Username:  username,
        Password:  password,
        Transport: &http.Transport{
            TLSClientConfig: &tls.Config{
                InsecureSkipVerify: true,
            },
        },
    })
    if err != nil {
        fmt.Println("Error creating OpenSearch client:", err)
        return
    }

    // Update a document
    indexName := "dev-article"
    documentID := "1"
    updatedFields := map[string]interface{}{
        "doc": map[string]interface{}{
            "content": "Updated api content-- OpenSearch is a powerful open-source search",
        },
    }

    err = updateDocument(client, indexName, documentID, updatedFields)
    if err != nil {
        fmt.Println("Error updating document:", err)
        return
    }
    fmt.Println("Document updated:", documentID)
}

func updateDocument(client *opensearch.Client, indexName string, documentID string, updatedFields map[string]interface{}) error {
    res, err := client.Update(indexName, documentID, opensearchutil.NewJSONReader(updatedFields))

    if err != nil {
        return err
    }
    defer res.Body.Close()

    if res.IsError() {
        return fmt.Errorf("update document request failed: %s", res.String())
    }

    return nil
}

Output:

Document updated: 1

Read Output After Update:

Retrieved Document: map[_id:1 _index:dev-article _primary_term:1 _seq_no:1 _source:map[content:Updated api content-- OpenSearch is a powerful open-source search title:Getting Started with OpenSearch] _version:2 found:true]

Deleting a Document:

To delete a document from OpenSearch, we can use the Delete API. Here's an example:

package main

import (
    "crypto/tls"
    "fmt"
    "net/http"

    opensearch "github.com/opensearch-project/opensearch-go/v2"
)

func main() {
    // Replace with your OpenSearch cluster details
    endpoint := "https://localhost:9200"
    username := "admin" // Leave empty if not using authentication
    password := "admin" // Leave empty if not using authentication

    // Create a client
    client, err := opensearch.NewClient(opensearch.Config{
        Addresses: []string{endpoint},
        Username:  username,
        Password:  password,
        Transport: &http.Transport{
            TLSClientConfig: &tls.Config{
                InsecureSkipVerify: true,
            },
        },
    })
    if err != nil {
        fmt.Println("Error creating OpenSearch client:", err)
        return
    }

    // Delete a document
    indexName := "dev-article"
    documentID := "1"
    err = deleteDocument(client, indexName, documentID)
    if err != nil {
        fmt.Println("Error deleting document:", err)
        return
    }
    fmt.Println("Document deleted:", documentID)
}

func deleteDocument(client *opensearch.Client, indexName string, documentID string) error {

    _, err := client.Delete(indexName, documentID)
    if err != nil {
        return err
    }

    return nil
}

Output:

Document deleted: 1

Conclusion:

In this article, we explored how to perform CRUD operations in OpenSearch using Go and the OpenSearch Go SDK. We covered the steps to create an index, index a document, retrieve a document, update a document, and delete a document. By leveraging the OpenSearch Go SDK, Go developers can interact with OpenSearch programmatically and build powerful search and analytics applications. Remember to customize the code examples according to your specific OpenSearch cluster configuration and requirements. Happy indexing and searching with OpenSearch and Go!