DEV Community

Cover image for Building a price comparison tool for Ghanaian shops. The hard part wasn't fetching prices, it was matching them
Elliot Brenya sarfo
Elliot Brenya sarfo

Posted on

Building a price comparison tool for Ghanaian shops. The hard part wasn't fetching prices, it was matching them

In Ghana, the same phone can cost very different amounts depending on where you look. Electronics shops have their own websites, marketplaces have thousands of individual sellers, and nobody puts the prices side by side.

So at TrustCheck, where I'm co-founder, we built Price Check: type "iPhone 15" or "air conditioner" and see the price at several Ghanaian shops at once, with links to each listing.

I assumed fetching the prices would be the hard part. It wasn't. The hard part was answering a question that sounds trivial: is this product actually what the person searched for?

Here's what I learned building it in Laravel 12 on plain shared hosting.

Reading each shop the way its own website does

Every shop is built differently, so there's one small parser per platform:

  • Shopify shops expose a public predictive-search feed and a /products.json catalogue.
  • WooCommerce shops usually have their JSON API closed, so we read the search results page and parse the product blocks.
  • Custom shops often load a whole catalogue as JSON and filter it in the browser, so we do the same: download it, cache it, search our copy.

One rule I set early: if a shop blocks automated requests, we don't fight it. We link to that shop's own search page instead. One major Ghanaian marketplace blocks automated requests, so it's link-only. That keeps the tool on good terms with the shops it sends customers to.

Shopify's 10-result ceiling

Shopify's predictive search returns at most 10 products. Search "laptop" and you get 10 laptops from a shop that sells 200. Comparing the lowest price on 10 random items is meaningless.

The fix was to stop searching live at all. A scheduled job downloads the shop's full catalogue every 4 hours, and visitors' searches only ever read the cached copy:

public function warmShopifyCatalogue(string $shop): int
{
    $items = [];
    for ($page = 1; $page <= 15; $page++) {
        $response = Http::timeout(25)->connectTimeout(8)
            ->get($this->storeUrl($shop) . '/products.json', ['limit' => 250, 'page' => $page]);

        $products = $response->successful() ? $response->json('products') : null;
        if (!is_array($products)) {
            throw new \RuntimeException("{$shop} catalogue page {$page}: HTTP {$response->status()}");
        }

        foreach ($products as $p) {

            $available = array_filter($p['variants'] ?? [], fn ($v) => ($v['available'] ?? true) !== false);
            $prices = array_filter(array_map(fn ($v) => (float) ($v['price'] ?? 0), $available));
            if (!$prices || empty($p['title'])) {
                continue;
            }
            $items[] = ['title' => $p['title'], 'handle' => $p['handle'], 'price' => min($prices)];
        }

        if (count($products) < 250) {
            break;          // last page
        }
        usleep(300000);     // be polite between pages
    }

    Cache::put("price-check:shopify:{$shop}", $items, 12 * 3600);

    return count($items);
}
Enter fullscreen mode Exit fullscreen mode

The cached copy lives for 12 hours while the refresh runs every 4, so one failed refresh never leaves us empty. If the cache is cold, the search falls back to the 10-item quick search, so a visitor never waits for a full download.

Two side benefits: searches are fast because they never touch the shop, and the shops get a handful of requests a day instead of one per visitor.

The matching problem

The first version matched on "every word of the query appears in the title." It failed in four different ways.

1. Shops and shoppers use different words

People search "TV". One shop's TV titles say "55 inch UHD LED" and never contain "TV". People in Ghana search "stove". Shops list a "4 Burner Gas Cooker". The top search on the whole tool was "air conditioning", while shops write "Air Condition" or "Air Conditioner".

So there's a hand-made synonym list, driven by what people actually search for:

private const SYNONYMS = [
    'tv'      => ['television', 'led', 'qled', 'oled', 'uhd'],
    'fridge'  => ['refrigerator', 'freezer'],
    'stove'   => ['cooker', 'burner', 'hob', 'cooktop'],
    'ac'      => ['conditioner', 'condition', 'conditioning', 'aircon'],
    // ...
];
Enter fullscreen mode Exit fullscreen mode

And some names get joined into one word before matching, on both the query and the titles, because people write them every possible way:

private const JOINS = [
    '/\bplay\s*station\b/' => 'playstation',
    '/\bps\s*([345])\b/'   => 'playstation $1',
    '/\bmac\s+book\b/'     => 'macbook',
    '/\bi\s+phone\b/'      => 'iphone',
];
Enter fullscreen mode Exit fullscreen mode

Lesson: log your searches. Our synonym list came straight from the most common queries that returned nothing.

2. Laptops that never say "laptop"

Shops name laptops by brand and specs only: Lenovo I5 (8/512ssd) V15 G4. No "laptop" anywhere. So for a laptop search, a title that has a laptop processor and a storage or memory spec counts as a laptop:

private static function looksLikeLaptop(string $title): bool
{
    $cpu  = '/\b(core\s*)?i[3579](-\d{3,5}\w*)?\b|\bryzen\b|\bceleron\b|\bcore\s+ultra\b/i';
    // Storage or memory, also when glued to the number ("8/512ssd", "512GBSSD")
    $spec = '/\b\d{1,2}\s*\/\s*\d{3,4}(?!\d)|\b\d{3,4}\s*gb|\b\d\s*tb\b|(ssd|hdd|ram)\b/i';
    $notLaptop = '/\b(all[\s-]*in[\s-]*one|aio|desktop|tower|monitor|mini\s*pc)\b/i';

    return preg_match($cpu, $title) && preg_match($spec, $title) && !preg_match($notLaptop, $title);
}
Enter fullscreen mode Exit fullscreen mode

Phones never list an i5 or a Ryzen, so they don't slip in. Desktops and all-in-ones are explicitly excluded.

3. You searched for a phone and got a phone case

"iPhone 13" matches "iPhone 13 Case", "iPhone 13 Screen Protector" and "Charger for iPhone 13". These are cheap, so they made the lowest price meaningless.

Accessories are filtered out unless the query is itself for an accessory:

private const ACCESSORY_WORDS = ['case', 'cover', 'protector', 'charger', 'cable',
    'adapter', 'strap', 'pouch', 'skin', 'film', 'holder', 'stand', 'mount', /* ... */];

private const ACCESSORY_EXCEPTIONS = [
    'screen'   => ['tv', 'television', 'monitor', 'projector', 'laptop'],
    'keyboard' => ['laptop', 'computer', 'pc', 'notebook', 'macbook', 'piano'],
];
Enter fullscreen mode Exit fullscreen mode

Words are compared without a trailing "s", so "Stands" and "Controllers" count too.

Then a second safety net: shops don't sell fakes, so if a shop item costs less than 15% of the median of all the results, it's almost certainly an accessory the word list missed, and it's dropped.

4. Free gifts in the title

A promotion titled "Galaxy S25 - Free TV" matched a search for TV. So anything from the word "free" to the end of the title is stripped before matching:

$title = preg_replace('/(\(|\+|-|–|\bplus\b|\bwith\b)?\s*\bfree\b.*$/iu', '', $offer['title']);
Enter fullscreen mode Exit fullscreen mode

Flagging listings that are too cheap to be real

Marketplace sellers are individuals, and some "brand new iPhone at a quarter of the price" listings are bait. So the tool compares each marketplace listing with the median price across the real shops for the same search:

$offer['suspicious'] = $isMarketplace
    && !$isUsedOrRefurbished
    && $offer['price'] < $shopMedian * 0.25;
Enter fullscreen mode Exit fullscreen mode

The guard rails took as much thought as the rule:

  • We only flag when at least two shops have prices, so there's a real median to compare against.
  • We only flag things worth faking: if the shop median is under GH₵1,000, or the search is for accessories, nothing gets flagged.
  • Used and refurbished listings are never flagged, because cheap is expected there.
  • Flagged listings aren't hidden. They're moved below the rest with a note. We never call a seller a scammer; we say the price is far below what shops charge.

Small things that protect the site

Never trust a link from a scraped page. Product URLs come from the shops' own data. If a shop page were compromised, it could slip a javascript: URL or a phishing link onto our site. So every link must be https and on that shop's own domain, or it's dropped:

public static function shopUrl(string $store, string $url): ?string
{
    $shopHost = preg_replace('/^www\./', '', parse_url(self::STORES[$store]['url'], PHP_URL_HOST));
    $parts = parse_url($url);
    $host = strtolower($parts['host'] ?? '');

    $ok = ($parts['scheme'] ?? '') === 'https'
        && ($host === $shopHost || str_ends_with($host, '.' . $shopHost))
        && filter_var($url, FILTER_VALIDATE_URL);

    return $ok ? $url : null;
}
Enter fullscreen mode Exit fullscreen mode

Rate-limit the whole site, not just each visitor. Per-IP limits don't stop someone with many addresses from turning your tool into a way to hammer the shops. There's a site-wide cap of 600 fresh searches an hour. Cached searches are free and don't count.

Don't cache failures or empty results. If every shop failed, it's almost certainly our connection, not "no results", and that shouldn't be stored for 3 hours. Random junk searches that find nothing aren't cached either, or they'd slowly fill the file cache, which never cleans up old entries on its own.

What I'd tell another developer

  1. Matching is the product. Anyone can fetch prices. Showing the right products is what makes the comparison trustworthy.
  2. Build your synonym list from real search logs, not from guessing.
  3. Use more than one safety net. A word list catches most accessories, and a price-based floor catches the rest.
  4. Sanitise everything that comes from someone else's page, especially URLs you render as links.
  5. Respect the sources. Cache catalogues, pause between pages, and link out to shops that don't want to be fetched.

Price Check is free at trustcheckapp.com/tools/price-check. If you've built product matching across stores, I'd love to know how you handled the accessory problem.

I'm Elliot, co-founder of TrustCheck, an enterprise technical writer and a researcher. I write about what I learn building software for the Ghanaian market.

Top comments (0)