Alain Airom (Ayrom)

Posted on May 13

🤖 Bob’s Guide to Google Genkit

#bob #ibmbob #genkit #go

Industrializing Generative AI with Go langauge!

Introduction

While the vast majority of generative AI applications are currently forged in Python — a language that has served the community exceptionally well — a growing contingent of developers is asking: what if we want to leverage the performance and concurrency of a language like Go? My discovery of Genkit, an open-source framework by Google designed specifically to streamline AI orchestration for Go developers. By utilizing the samples and documentation provided by the project, I set out to build anapplication using my automation buddy, Bob. A standout feature that accelerated this process was the “skills” implementation found in the Genkit GitHub repository; by providing these skills “as-is” to Bob, I was able to bypass significant manual configuration, effectively fast-tracking the development cycle and ensuring the resulting application and the documentation presented below remained straightforward and robust.

Genkit presentation (Excerpt from GitHub)

Genkit is an open-source framework for building full-stack AI-powered applications, built and used in production by Google’s Firebase. It provides SDKs for multiple programming languages with varying levels of stability:

JavaScript/TypeScript: Production-ready with full feature support
Go: Production-ready with full feature support
Python (Beta): Wide feature support approaching production-readiness
Dart (Preview): Early development with core functionality
It offers a unified interface for integrating AI models from providers like Google, OpenAI, Anthropic, Ollama, and more. Rapidly build and deploy production-ready chatbots, automations, and recommendation systems using streamlined APIs for multimodal content, structured outputs, tool calling, and agentic workflows.

🚀 Why Genkit for Go?

As mentioned in the introduction, while Python is often used for AI research (for many good reasons by the way), Go is the language of infrastructure. Bob leverages Genkit to bring AI directly into the SDLC with:

Typed Flows: AI logic is defined as standard, type-safe Go functions.
Built-in Observability: OpenTelemetry is integrated from the start, not added as an afterthought.

The Implementation of The Code

So based on the toolkit and the samples provided Bob built an application with a UI, leaving the choice to locall Ollama LLM list avaiable to the user and a built-in OpenTelemtry dashboard.

💻 Real-Time Interface: Genkit & LLM Handling

// Initialize Genkit with the Ollama plugin
g = genkit.Init(ctx,
    genkit.WithPlugins(otelPlugin, ollamaPlugin),
)

// Dynamic model discovery from local Ollama tags
func discoverAndDefineModels() error {
    resp, err := http.Get(ollamaHost + "/api/tags")
    // ... decoding logic ...
    for _, model := range modelsResp.Models {
        modelName := "ollama/" + model.Name
        availableModels = append(availableModels, modelName)
    }
    return nil
}

// Defining a Genkit Streaming Flow
streamFlow = genkit.DefineStreamingFlow(g, "chatStream",
    func(ctx context.Context, input ChatRequest, callback core.StreamCallback[string]) (string, error) {
        stream := genkit.GenerateStream(ctx, g,
            ai.WithModelName(input.Model),
            ai.WithPrompt(input.Message),
        )
        // ... iteration logic to handle chunks ...
    },
)

The application features a sleek, chat interface which provides:

Real-time Streaming: Responses are delivered chunk-by-chunk for a smooth user experience.
Dynamic Model Discovery: The discoverAndDefineModels function queries the Ollama /api/tags endpoint to fetch all locally installed models. These are then registered within the Genkit ecosystem using the ollama/ prefix.
Connection Status: Visual indicators to ensure the local inference server is active.
Plugin Initialization: The application initializes the Genkit core by registering the Ollama plugin, which is configured to communicate with a local Ollama server (defaulting to http://localhost:11434).
Prompt Management: Each flow dynamically selects the model requested by the user or falls back to a default (e.g., ollama/llama3.2:latest) to process the incoming ChatRequest.
Streaming Flow: Uses genkit.DefineStreamingFlow and genkit.GenerateStreamto handle real-time response generation, allowing the UI to display text as it is generated.

🔭 Full-Stack Observability — OpenTelemetry Implementation

The application tracks and monitors AI performance using the OpenTelemetry standard.

Provider Setup: The application uses the genkit-opentelemetry-go plugin to initialize a specialized provider. For development, it uses PresetConsole, which outputs all telemetry data directly to the terminal.
Service Configuration: The telemetry is tagged with the service name "genkit-ollama-app" and is configured to force-export data even in development environments to ensure visibility.
Automatic Instrumentation: Because the OpenTelemetry plugin is registered during genkit.Init, all subsequent actions—such as flow execution and model inference—are automatically wrapped in spans.
Tracing: Each request generates a unique Trace ID and multiple Span IDs that capture the start time, end time, and any errors encountered during the AI’s execution.
Metrics: The system collects specific metrics like genkit/action/requests to count total interactions and genkit/action/latency to measure performance in milliseconds.
Telemetry Dashboard: A dedicated HTTP handler (/telemetry) serves a dashboard that provides a visual overview of the active telemetry status, service identifiers, and registered models.

// Initialize OpenTelemetry with a console-export preset
otelPlugin := opentelemetry.NewWithPreset(opentelemetry.PresetConsole, opentelemetry.Config{
    ServiceName: "genkit-ollama-app",
    LogLevel:    slog.LevelInfo,
    ForceExport: true, // Ensure telemetry is generated in dev
})

// The Telemetry Dashboard Handler (Excerpt)
func handleTelemetryDashboard(w http.ResponseWriter, r *http.Request) {
    html := `<!DOCTYPE html>
    <html>
    <head><title>OpenTelemetry Dashboard</title></head>
    <body>
        <h1>🔭 OpenTelemetry Dashboard</h1>
        <div class="info-card">
            <h3>Metrics Collected:</h3>
            <ul>
                <li>genkit/action/requests - Total number of action requests</li>
                <li>genkit/action/latency - Action execution latency in milliseconds</li>
            </ul>
        </div>
        </body>
    </html>`
    w.Header().Set("Content-Type", "text/html")
    w.Write([]byte(html))
}

Conclusion

To conclude this exploration of modern AI orchestration, the sample code developed by IBM SDLC Bob serves as a powerful testament to the viability of building generative AI applications outside the traditional Python ecosystem. By leveraging Google Genkit’s Go SDK, the project showcases a seamless integration of LLMs inferencewith enterprise-grade observability provided by OpenTelemetry. The implementation highlights the ease with which developers can define complex, traceable “Flows” for both standard and real-time streaming chat or other type of generative ai applications interfaces.

Ultimately, this proves that using Go for AI development does not mean sacrificing developer experience or performance.

By adopting the “skills” provided in the Genkit repository, Bob was able to fast-track the creation of a robust, production-ready service — complete with a sleek dark-mode UI and comprehensive telemetry dashboards.

>>> Thanks for reading 🤠 <<<

DEV Community