Serving Mandi Prices at the Edge: A Low-Latency Strategy for Rural India
Building for the Indian agricultural sector means fighting a different battle than typical web development. Your primary adversary isn't a competitor—it’s the network. In rural mandis, a 1,500ms round-trip time over unstable 4G is a dealbreaker.
At KhetiSetu, we’ve dedicated significant engineering effort to optimizing how Mandi Bhav (commodity prices) reaches farmers. Imagine a farmer in Unnao or Mandsaur negotiating with an Arhatiya (trader). If they have to watch a loading spinner for ten seconds while our Mumbai-based server queries a distant database, the deal is likely lost. They need the current price of mustard or wheat instantly.
The challenge with mandi data is twofold: it is geographically hyper-specific and highly volatile within short windows. Prices surge during morning arrivals and stabilize in the afternoon. Hitting government APIs or running scrapers on every single user request is an architectural dead end.
Architecting for Low Latency
In a standard web setup, a request travels from the user’s phone to the origin server. In rural India, where "last-mile" connectivity is often unreliable, the only solution is to bring the data closer to the user. We achieve this by leveraging the Edge.
Our strategy combines Incremental Static Regeneration (ISR) with Edge Middleware. This ensures data is fresh enough to be accurate but served from a CDN node in a nearby city, rather than a central data center.
Data Flow Overview
graph LR
A[Farmer App] --> B[Edge CDN Node]
B -- Cache Hit --> A
B -- Cache Miss/Stale --> C[Next.js ISR Worker]
C --> D[Redis Cache]
D --> E[Mandi Data Source / API]
E --> D
D --> C
C --> B
By pushing logic to the edge, we drastically reduce Time to First Byte (TTFB). A farmer in Punjab hitting a Point of Presence (POP) in Delhi will always outperform one hitting a server in a distant region.
Implementing ISR for Commodity Prices
Next.js offers robust tools for this via ISR. Mandi prices don’t change every second; updates typically occur every 30 minutes to an hour. Real-time WebSockets are overkill here. We need "fresh enough" data.
Here is a simplified view of our price-fetching logic for specific mandis.
// pages/mandi/[state]/[district]/[commodity].tsx
import { GetStaticProps, GetStaticPaths } from 'next';
export const getStaticProps: GetStaticProps = async (context) => {
const { state, district, commodity } = context.params!;
try {
// Fetching from our internal optimized microservice
const res = await fetch(`https://api.khetisetu.com/v1/prices/${state}/${district}/${commodity}`, {
next: { revalidate: 1800 } // Revalidate every 30 minutes
});
const data = await res.json();
if (!data) {
return { notFound: true };
}
return {
props: {
priceData: data,
lastUpdated: new Date().toISOString(),
},
// Ensure the page is regenerated at most every 30 minutes
revalidate: 1800,
};
} catch (error) {
console.error("Mandi Data Fetch Error:", error);
return { props: { error: true }, revalidate: 60 };
}
};
Why 1800 seconds?
The revalidate timer is a balancing act. Set it too low, and you hammer upstream sources (like Agmarknet or private scrapers), spiking compute costs. Set it too high, and farmers see stale prices that may have been superseded by mid-day corrections. Through field testing, we found 30 minutes to be the sweet spot.
Managing the Vernacular and Geographic Long-Tail
India has over 7,000 regulated mandis. Multiply that by hundreds of commodities and varieties (e.g., Desi vs. Hybrid Maize), and you get a massive matrix of potential URLs.
Pre-rendering every combination is impossible—it would explode build times. Instead, we use a "blocking" fallback strategy. When a farmer queries a niche mandi in remote Odisha that hasn’t been cached yet, the first request will be slower as the server generates the page. However, every subsequent user in that region gets the cached version instantly.
| Strategy | Latency (2G/3G) | Server Load | Data Freshness |
|---|---|---|---|
| Client-side Fetch | 3000ms+ | High | Real-time |
| Server-side Rendering | 1500ms - 2000ms | High | Real-time |
| Edge Caching (ISR) | 200ms - 500ms | Low | ~30 mins old |
The "Stale-While-Revalidate" UX
A critical nuance in rural UX is managing perception. If a page takes two seconds to load, users may assume the app is broken and close it.
We mitigate this using the stale-while-revalidate pattern: we serve the old price data immediately while a background process fetches the new data. Crucially, we explicitly display the "Last Updated" timestamp.
In our UI, we use a simple color-coded indicator:
- Green: Updated less than 1 hour ago.
- Yellow: Updated 1-4 hours ago.
- Red/Grey: Data is from yesterday (common for mandis that haven't reported yet).
This transparency builds trust. At KhetiSetu, we’ve found that farmers value the "Last Updated" timestamp almost as much as the price itself. It clarifies whether they are looking at this morning’s auction Bhav or yesterday’s closing rate.
Infrastructure Lessons from the Field
Initially, we relied on a single global cache. We quickly learned that "Average Latency" is a misleading metric. Your average might look impressive due to users in Bangalore on fiber, but the P99 latency for a farmer in rural Bihar is what actually drives retention.
We shifted to a regionalized edge strategy and implemented a "Cache Warming" script for the top 500 most active mandis. Every morning at 10:00 AM (when the first major price bulletins are released), we proactively populate the edge cache for high-traffic locations. This ensures that by the time the market crowd arrives, the data is already sitting at the edge, ready to be served in milliseconds.
Top comments (0)