How to Create a Web Search Tool Using OpenAI API in Go

In this tutorial, we will build a simple web search tool in Go using the OpenAI API. This application will utilize OpenAI's capabilities to process search queries and integrate Google Search for retrieving relevant results. By the end of this guide, you will have a functional web search tool to enhance your projects.

---

Table of Contents

1. Introduction
2. Prerequisites
3. Folder Structure
4. Setting Up the Project
5. Writing the Search Logic
6. Testing the Application
7. Conclusion

---

Introduction

Web search tools powered by OpenAI offer an intelligent way to interact with search engines and process results. In this tutorial, we will build a Go application that:

1. Accepts user queries.
2. Retrieves search results using Google Search.
3. Processes the results with OpenAI API for summarization.

---

Prerequisites

Before starting, ensure you have:

• Go installed (version 1.19 or higher).
• An OpenAI API key.
• Access to a Google Search API or a suitable browser tool.
• Basic knowledge of Go programming.

---

Folder Structure

Create the following structure for your project:

websearch-go/
|-- main.go
|-- search/
|   |-- search.go
|-- openai/
    |-- openai.go

---

Setting Up the Project

Step 1: Initialize the Project

Run the following command to initialize a new Go module:

mkdir websearch-go && cd websearch-go
go mod init websearch-go

Step 2: Install Dependencies

We will use the following libraries:

• github.com/joho/godotenv for environment variables.
• net/http for HTTP requests.
• encoding/json for parsing JSON responses.

Install the required libraries:

go get github.com/joho/godotenv

Step 3: Create a .env File

Add your OpenAI API key and Google API key to a .env file:

OPENAI_API_KEY=your_openai_api_key
GOOGLE_API_KEY=your_google_api_key
GOOGLE_SEARCH_ENGINE_ID=your_search_engine_id

---

Writing the Search Logic

Step 1: Implement the Google Search

Create a new file search/search.go:

package search

import (
    "encoding/json"
    "fmt"
    "net/http"
    "os"
)

type GoogleSearchResult struct {
    Items []struct {
        Title       string `json:"title"`
        Link        string `json:"link"`
        Snippet     string `json:"snippet"`
    } `json:"items"`
}

func GoogleSearch(query string) ([]string, error) {
    apiKey := os.Getenv("GOOGLE_API_KEY")
    cx := os.Getenv("GOOGLE_SEARCH_ENGINE_ID")
    url := fmt.Sprintf("https://www.googleapis.com/customsearch/v1?q=%s&key=%s&cx=%s", query, apiKey, cx)

    resp, err := http.Get(url)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()

    var result GoogleSearchResult
    if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
        return nil, err
    }

    var links []string
    for _, item := range result.Items {
        links = append(links, fmt.Sprintf("%s: %s", item.Title, item.Link))
    }

    return links, nil
}

Step 2: Integrate OpenAI

Create a new file openai/openai.go:

package openai

import (
    "bytes"
    "encoding/json"
    "fmt"
    "net/http"
    "os"
)

type OpenAIRequest struct {
    Model       string   `json:"model"`
    Prompt      string   `json:"prompt"`
    MaxTokens   int      `json:"max_tokens"`
    Temperature float64  `json:"temperature"`
}

type OpenAIResponse struct {
    Choices []struct {
        Text string `json:"text"`
    } `json:"choices"`
}

func Summarize(prompt string) (string, error) {
    apiKey := os.Getenv("OPENAI_API_KEY")
    url := "https://api.openai.com/v1/completions"

    requestBody := OpenAIRequest{
        Model:       "text-davinci-003",
        Prompt:      prompt,
        MaxTokens:   150,
        Temperature: 0.7,
    }

    jsonBody, err := json.Marshal(requestBody)
    if err != nil {
        return "", err
    }

    req, err := http.NewRequest("POST", url, bytes.NewBuffer(jsonBody))
    if err != nil {
        return "", err
    }

    req.Header.Set("Content-Type", "application/json")
    req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", apiKey))

    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        return "", err
    }
    defer resp.Body.Close()

    var response OpenAIResponse
    if err := json.NewDecoder(resp.Body).Decode(&response); err != nil {
        return "", err
    }

    if len(response.Choices) > 0 {
        return response.Choices[0].Text, nil
    }

    return "", fmt.Errorf("no response from OpenAI")
}

Step 3: Combine Google Search and OpenAI

In main.go, combine both services:

package main

import (
    "fmt"
    "log"
    "os"

    "websearch-go/openai"
    "websearch-go/search"

    "github.com/joho/godotenv"
)

func main() {
    if err := godotenv.Load(); err != nil {
        log.Fatal("Error loading .env file")
    }

    query := "Latest technology trends"
    fmt.Println("Searching for:", query)

    results, err := search.GoogleSearch(query)
    if err != nil {
        log.Fatal(err)
    }

    for _, result := range results {
        fmt.Println("Result:", result)
    }

    summary, err := openai.Summarize("Summarize these results: \n" + fmt.Sprint(results))
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("Summary:", summary)
}

---

Testing the Application

Run the application using:

go run main.go

You should see search results followed by a summarized output.

---

Conclusion

In this guide, we created a simple web search tool using Go, OpenAI API, and Google Search. This combination allows us to fetch and process search results intelligently, making it a powerful tool for various applications. Feel free to expand on this by adding a web interface or additional features!