DEV Community

Cover image for Agentic farm advisory assistant built with Gemma 4 + Google AI Studio
Agbo, Daniel Onuoha
Agbo, Daniel Onuoha

Posted on

Agentic farm advisory assistant built with Gemma 4 + Google AI Studio

#ai #machinelearning #gemma #javascript

We will walk through a complete, working project: an agentic farm advisory assistant built with Gemma 4 through Google AI Studio's Gemini API. It diagnoses crop issues from photos, checks weather-based planting windows, and logs farm activity through real function-calling — prototyped in the browser, then shipped as an Express backend with a chat UI.

What We're Building

An advisory chatbot for smallholder farmers and agro-logistics platforms that can:

  • Diagnose a crop disease or pest issue from an uploaded photo
  • Check whether current weather conditions are safe for planting or spraying
  • Look up market prices for a given crop
  • Log a farm activity (planting, spraying, harvest) to a farmer's record
  • Reply naturally, including in Nigerian Pidgin or local phrasing

The model never guesses market prices or weather data — every factual answer comes from an actual function call against a backend, not the model's own assumptions.

Step 1: Prototype the Agent in Google AI Studio

Before writing any code, the entire agent was designed inside aistudio.google.com:

  1. Opened the model picker and selected gemma-4-31b-it for its multimodal (image) support
  2. Added this system instruction in the chat panel: 
You are a friendly, practical farm advisory assistant for smallholder farmers
in Nigeria. Always use the provided tools for weather checks, market prices,
and activity logging — never guess prices or weather data. When a farmer
uploads a crop photo, examine it carefully before giving diagnosis and
next steps. Keep responses short, practical, and in plain language. Reply
in the same language or Pidgin the farmer writes in.
Enter fullscreen mode Exit fullscreen mode
  1. Defined four tools in the Tools panel — check_weather_window, get_market_price, log_farm_activity, and diagnose_crop_image — each with a JSON schema and a scoped description
  2. Uploaded sample crop photos (a leaf with brown spots, a tomato plant with wilting) directly in the AI Studio chat to test multimodal diagnosis before writing any code
  3. Tested prompts like "Is it safe to spray my maize today?" until the model reliably called check_weather_window instead of answering from general knowledge
  4. Clicked Get Code to export a starting JavaScript snippet using @google/genai

Testing the image diagnosis directly in the browser first was the most valuable step — it's far easier to spot a vague description problem ("model just said 'looks unhealthy'") in a live chat than after it's buried in server logs.

Step 2: Define the Tools and Mock Backend 

// tools.js
export const tools = [{
  functionDeclarations: [
    {
      name: "check_weather_window",
      description: "Use when the farmer asks if it's safe or a good time to plant, spray, or harvest. Never guess weather.",
      parameters: {
        type: "OBJECT",
        properties: {
          location: { type: "STRING", description: "e.g. Port Harcourt, Owerri" },
          activity: { type: "STRING", description: "planting, spraying, or harvesting" }
        },
        required: ["location", "activity"]
      }
    },
    {
      name: "get_market_price",
      description: "Use ONLY when the farmer asks for the current price of a crop. Never guess a price.",
      parameters: {
        type: "OBJECT",
        properties: {
          crop: { type: "STRING", description: "e.g. cassava, maize, tomato" },
          market: { type: "STRING", description: "e.g. Mile 1 Market, Port Harcourt" }
        },
        required: ["crop"]
      }
    },
    {
      name: "log_farm_activity",
      description: "Use when the farmer reports completing an activity like planting, spraying, or harvesting.",
      parameters: {
        type: "OBJECT",
        properties: {
          farmerId: { type: "STRING" },
          activity: { type: "STRING", description: "planting, spraying, harvesting" },
          crop: { type: "STRING" },
          notes: { type: "STRING" }
        },
        required: ["farmerId", "activity", "crop"]
      }
    },
    {
      name: "diagnose_crop_image",
      description: "Use when the farmer uploads a photo of a crop showing signs of disease, pest damage, or poor health.",
      parameters: {
        type: "OBJECT",
        properties: {
          crop: { type: "STRING", description: "e.g. maize, tomato, cassava" },
          symptomDescription: { type: "STRING", description: "Visible symptoms described from the image" }
        },
        required: ["crop", "symptomDescription"]
      }
    }
  ]
}];

const weatherData = {
  "port harcourt": { rainChance: 20, condition: "clear, light wind" },
  "owerri": { rainChance: 70, condition: "heavy rain expected" }
};

const marketPrices = {
  cassava: { pricePerBag: 18500, currency: "NGN", market: "Mile 1 Market" },
  maize: { pricePerBag: 22000, currency: "NGN", market: "Mile 1 Market" },
  tomato: { pricePerBasket: 15000, currency: "NGN", market: "Mile 1 Market" }
};

const activityLog = [];

export async function check_weather_window({ location, activity }) {
  const key = location.toLowerCase();
  const weather = weatherData[key];
  if (!weather) return { error: "Location not found in weather data" };

  const safe = activity === "spraying" ? weather.rainChance < 40 : true;
  return { location, activity, ...weather, recommendation: safe ? "Safe to proceed" : "Wait — rain expected, risk of runoff" };
}

export async function get_market_price({ crop, market }) {
  const price = marketPrices[crop.toLowerCase()];
  return price ? { crop, ...price } : { error: `No price data for ${crop}` };
}

export async function log_farm_activity({ farmerId, activity, crop, notes }) {
  const entry = { farmerId, activity, crop, notes: notes || "", date: "2026-07-07" };
  activityLog.push(entry);
  return { logged: true, ...entry };
}

export async function diagnose_crop_image({ crop, symptomDescription }) {
  // In production, this would call a vision model or trained classifier.
  // Here Gemma 4's own multimodal reasoning already produced symptomDescription
  // from the uploaded image before calling this tool.
  const knownIssues = {
    "brown spots on leaves": "Likely leaf blight — recommend copper-based fungicide, improve drainage",
    "wilting despite watering": "Possible bacterial wilt or root rot — check soil drainage and remove affected plants"
  };
  const match = Object.keys(knownIssues).find(k => symptomDescription.toLowerCase().includes(k.split(" ")[0]));
  return {
    crop,
    diagnosis: match ? knownIssues[match] : "Symptoms noted but inconclusive — recommend local extension officer visit",
  };
}

export const toolFunctions = { check_weather_window, get_market_price, log_farm_activity, diagnose_crop_image };
Enter fullscreen mode Exit fullscreen mode

Step 3: Build the Agent Loop in Express

The core of the backend is a loop that keeps resolving tool calls — including image-based diagnosis — until Gemma 4 returns a final plain-text answer:

// server.js
import express from "express";
import { GoogleGenAI } from "@google/genai";
import { tools, toolFunctions } from "./tools.js";

const app = express();
app.use(express.json({ limit: "10mb" })); // allow base64 image payloads

const client = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });
const MODEL = "gemma-4-31b-it";
const MAX_STEPS = 4;

const SYSTEM_INSTRUCTION = `You are a friendly, practical farm advisory assistant...`;
const sessions = new Map();

app.post("/api/chat", async (req, res) => {
  const { sessionId = "default", message, imageBase64 } = req.body;

  if (!sessions.has(sessionId)) {
    sessions.set(sessionId, client.chats.create({
      model: MODEL,
      config: { systemInstruction: SYSTEM_INSTRUCTION, tools }
    }));
  }
  const chat = sessions.get(sessionId);

  const parts = imageBase64
    ? [{ inlineData: { mimeType: "image/jpeg", data: imageBase64 } }, { text: message }]
    : message;

  let response = await chat.sendMessage({ message: parts });
  let steps = 0;

  while (response.functionCalls?.length && steps < MAX_STEPS) {
    const call = response.functionCalls[0];
    const result = await toolFunctions[call.name](call.args);
    response = await chat.sendMessage({
      message: [{ functionResponse: { name: call.name, response: result } }]
    });
    steps += 1;
  }

  res.json({ reply: response.text, toolStepsUsed: steps });
});

app.listen(3000, () => console.log("Agro advisory agent running on port 3000"));
Enter fullscreen mode Exit fullscreen mode

Step 4: Run It — Actual Results

With the server running (npm start) and a GEMINI_API_KEY from AI Studio set in .env, here's what real requests return.

Request — weather check before spraying:

curl -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"sessionId":"farmer1","message":"Is it safe to spray my maize in Owerri today?"}'
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "reply": "Not today — Owerri has a 70% chance of rain, which could wash off the spray before it works. Wait for a drier day.",
  "toolStepsUsed": 1
}
Enter fullscreen mode Exit fullscreen mode

Request — market price check:

curl -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"sessionId":"farmer1","message":"How much is cassava selling for now?"}'
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "reply": "Cassava is currently going for NGN 18,500 per bag at Mile 1 Market.",
  "toolStepsUsed": 1
}
Enter fullscreen mode Exit fullscreen mode

Request — crop photo diagnosis (image + text):

curl -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"sessionId":"farmer1","message":"My tomato plant looks sick, see photo","imageBase64":"<base64 jpeg data>"}'
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "reply": "I see wilting and yellowing leaves on your tomato plant. This looks like bacterial wilt or root rot. Check your soil drainage and remove the worst-affected plants to stop it spreading.",
  "toolStepsUsed": 1
}
Enter fullscreen mode Exit fullscreen mode

Request — Pidgin, activity logging:

curl -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"sessionId":"farmer1","message":"I don finish plant my maize for farm today, farmer ID FARM-002"}'
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "reply": "Good work! I don log say you plant maize today for your farm record (FARM-002). E dey saved.",
  "toolStepsUsed": 1
}
Enter fullscreen mode Exit fullscreen mode

Why This Pattern Works

  • Correctness: weather windows and market prices come from toolFunctions, not the model's own guess, so a farmer never gets rain-safety advice invented on the spot
  • Multimodal grounding: Gemma 4 reads the uploaded photo directly, describes the symptoms, and hands that description to a diagnosis tool — combining visual reasoning with a controllable, auditable backend step
  • Traceability: toolStepsUsed on every response makes it easy to log exactly what the agent did, useful for tracking advisory accuracy over a growing season
  • Fast iteration: because the tool descriptions and image-handling behavior were validated in AI Studio's chat first, the Express implementation worked correctly on the first real run

Taking It Further

Swap the mock weather and price data in tools.js for a real weather API and a live market-price feed (e.g., from a state agriculture board or a partner logistics platform), move the activity log from an in-memory array to MongoDB, and replace the rule-based diagnose_crop_image matching with a fine-tuned vision classifier once you have enough labeled crop-disease photos. For farmers in low-connectivity rural areas, consider porting the same tool schema to a self-hosted E2B/E4B deployment on an Android device or Jetson Orin Nano so diagnosis works even without a live network connection.

Top comments (0)