DEV Community

Cover image for How to Create a Web Search Tool Using OpenAI API in Go
Mazyar Yousefiniyae shad
Mazyar Yousefiniyae shad

Posted on

3 1 1 1 1

How to Create a Web Search Tool Using OpenAI API in Go

In this tutorial, we will build a simple web search tool in Go using the OpenAI API. This application will utilize OpenAI's capabilities to process search queries and integrate Google Search for retrieving relevant results. By the end of this guide, you will have a functional web search tool to enhance your projects.


Table of Contents

  1. Introduction
  2. Prerequisites
  3. Folder Structure
  4. Setting Up the Project
  5. Writing the Search Logic
  6. Testing the Application
  7. Conclusion

Introduction

Web search tools powered by OpenAI offer an intelligent way to interact with search engines and process results. In this tutorial, we will build a Go application that:

  1. Accepts user queries.
  2. Retrieves search results using Google Search.
  3. Processes the results with OpenAI API for summarization.

Prerequisites

Before starting, ensure you have:

  • Go installed (version 1.19 or higher).
  • An OpenAI API key.
  • Access to a Google Search API or a suitable browser tool.
  • Basic knowledge of Go programming.

Folder Structure

Create the following structure for your project:

websearch-go/
|-- main.go
|-- search/
|   |-- search.go
|-- openai/
    |-- openai.go
Enter fullscreen mode Exit fullscreen mode

Setting Up the Project

Step 1: Initialize the Project

Run the following command to initialize a new Go module:

mkdir websearch-go && cd websearch-go
go mod init websearch-go
Enter fullscreen mode Exit fullscreen mode

Step 2: Install Dependencies

We will use the following libraries:

  • github.com/joho/godotenv for environment variables.
  • net/http for HTTP requests.
  • encoding/json for parsing JSON responses.

Install the required libraries:

go get github.com/joho/godotenv
Enter fullscreen mode Exit fullscreen mode

Step 3: Create a .env File

Add your OpenAI API key and Google API key to a .env file:

OPENAI_API_KEY=your_openai_api_key
GOOGLE_API_KEY=your_google_api_key
GOOGLE_SEARCH_ENGINE_ID=your_search_engine_id
Enter fullscreen mode Exit fullscreen mode

Writing the Search Logic

Step 1: Implement the Google Search

Create a new file search/search.go:

package search

import (
    "encoding/json"
    "fmt"
    "net/http"
    "os"
)

type GoogleSearchResult struct {
    Items []struct {
        Title       string `json:"title"`
        Link        string `json:"link"`
        Snippet     string `json:"snippet"`
    } `json:"items"`
}

func GoogleSearch(query string) ([]string, error) {
    apiKey := os.Getenv("GOOGLE_API_KEY")
    cx := os.Getenv("GOOGLE_SEARCH_ENGINE_ID")
    url := fmt.Sprintf("https://www.googleapis.com/customsearch/v1?q=%s&key=%s&cx=%s", query, apiKey, cx)

    resp, err := http.Get(url)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()

    var result GoogleSearchResult
    if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
        return nil, err
    }

    var links []string
    for _, item := range result.Items {
        links = append(links, fmt.Sprintf("%s: %s", item.Title, item.Link))
    }

    return links, nil
}
Enter fullscreen mode Exit fullscreen mode

Step 2: Integrate OpenAI

Create a new file openai/openai.go:

package openai

import (
    "bytes"
    "encoding/json"
    "fmt"
    "net/http"
    "os"
)

type OpenAIRequest struct {
    Model       string   `json:"model"`
    Prompt      string   `json:"prompt"`
    MaxTokens   int      `json:"max_tokens"`
    Temperature float64  `json:"temperature"`
}

type OpenAIResponse struct {
    Choices []struct {
        Text string `json:"text"`
    } `json:"choices"`
}

func Summarize(prompt string) (string, error) {
    apiKey := os.Getenv("OPENAI_API_KEY")
    url := "https://api.openai.com/v1/completions"

    requestBody := OpenAIRequest{
        Model:       "text-davinci-003",
        Prompt:      prompt,
        MaxTokens:   150,
        Temperature: 0.7,
    }

    jsonBody, err := json.Marshal(requestBody)
    if err != nil {
        return "", err
    }

    req, err := http.NewRequest("POST", url, bytes.NewBuffer(jsonBody))
    if err != nil {
        return "", err
    }

    req.Header.Set("Content-Type", "application/json")
    req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", apiKey))

    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        return "", err
    }
    defer resp.Body.Close()

    var response OpenAIResponse
    if err := json.NewDecoder(resp.Body).Decode(&response); err != nil {
        return "", err
    }

    if len(response.Choices) > 0 {
        return response.Choices[0].Text, nil
    }

    return "", fmt.Errorf("no response from OpenAI")
}
Enter fullscreen mode Exit fullscreen mode

Step 3: Combine Google Search and OpenAI

In main.go, combine both services:

package main

import (
    "fmt"
    "log"
    "os"

    "websearch-go/openai"
    "websearch-go/search"

    "github.com/joho/godotenv"
)

func main() {
    if err := godotenv.Load(); err != nil {
        log.Fatal("Error loading .env file")
    }

    query := "Latest technology trends"
    fmt.Println("Searching for:", query)

    results, err := search.GoogleSearch(query)
    if err != nil {
        log.Fatal(err)
    }

    for _, result := range results {
        fmt.Println("Result:", result)
    }

    summary, err := openai.Summarize("Summarize these results: \n" + fmt.Sprint(results))
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("Summary:", summary)
}
Enter fullscreen mode Exit fullscreen mode

Testing the Application

Run the application using:

go run main.go
Enter fullscreen mode Exit fullscreen mode

You should see search results followed by a summarized output.


Conclusion

In this guide, we created a simple web search tool using Go, OpenAI API, and Google Search. This combination allows us to fetch and process search results intelligently, making it a powerful tool for various applications. Feel free to expand on this by adding a web interface or additional features!

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read more

Top comments (0)

The best way to debug slow web pages cover image

The best way to debug slow web pages

Tools like Page Speed Insights and Google Lighthouse are great for providing advice for front end performance issues. But what these tools can’t do, is evaluate performance across your entire stack of distributed services and applications.

Watch video