DEV Community

Cover image for Your Laravel App is Choking on Search Queries (Here's How to Fix It in 47 Lines)
Igor Nosatov
Igor Nosatov

Posted on

Your Laravel App is Choking on Search Queries (Here's How to Fix It in 47 Lines)

  • 💡 Manual aggregations in Laravel = slow queries + messy code + sad users
  • 🔥 Elasticsearch does in milliseconds what takes your foreach loops seconds
  • ✅ Real example: 2000+ lines → 47 lines, 8s → 180ms
  • ⚠️ You're probably over-engineering your search right now
  • 🎁 Copy-paste solution included below

The $127,000 Foreach Loop

Picture this: It's 2 AM. Your Laravel app just crashed. Again.

The culprit? A "simple" product search with faceted filters. You know, the kind every e-commerce site has. Categories on the left, price ranges, brands...

Your code looks something like this:

// RIP your database 💀
$products = Product::where('status', 'active')->get();
$categories = [];

foreach ($products as $product) {
    if (!isset($categories[$product->category_id])) {
        $categories[$product->category_id] = 0;
    }
    $categories[$product->category_id]++;
}

// Now do the same for brands... colors... sizes... 
// Your DB is crying
Enter fullscreen mode Exit fullscreen mode

The damage:

  • 8.3 seconds response time
  • 15,000 DB queries per search
  • Users bouncing faster than a rubber ball
  • Your AWS bill looking like a phone number

Sound familiar?

Why Developers Keep Making This Mistake

⚠️ Common trap #1: "I'll just add an index"

Narrator: The index didn't help.

⚠️ Common trap #2: "Let me cache this"

Cache invalidation has entered the chat

⚠️ Common trap #3: "Raw SQL will be faster"

You just wrote a query that would make your DBA cry

The real problem? You're using a relational database for a non-relational problem.

Aggregations, full-text search, and faceted filtering aren't what MySQL was designed for. It's like using a hammer to screw in a lightbulb—technically possible, but why?

Enter Elasticsearch: The Aggregation Beast

Elasticsearch treats aggregations as a first-class citizen. Not an afterthought. Not a "let me write 47 subqueries" situation.

The same feature in Elasticsearch:

use Elastic\Elasticsearch\ClientBuilder;

$client = ClientBuilder::create()->build();

$params = [
    'index' => 'products',
    'body' => [
        'query' => [
            'match' => ['status' => 'active']
        ],
        'aggs' => [
            'categories' => [
                'terms' => ['field' => 'category_id']
            ],
            'brands' => [
                'terms' => ['field' => 'brand.keyword']
            ],
            'price_ranges' => [
                'range' => [
                    'field' => 'price',
                    'ranges' => [
                        ['to' => 50],
                        ['from' => 50, 'to' => 100],
                        ['from' => 100]
                    ]
                ]
            ]
        ],
        'size' => 20
    ]
];

$response = $client->search($params);
Enter fullscreen mode Exit fullscreen mode

Response time: 180ms

Lines of code: 47

Your DBA's stress level: 📉

The Real-World Impact: A Case Study

Before Elasticsearch:

Search "red sneakers size 42"
├─ Query 1: Find products (2.3s)
├─ Query 2: Get categories (1.8s)
├─ Query 3: Get brands (1.5s)
├─ Query 4: Calculate price ranges (2.1s)
└─ Query 5: Apply filters (0.6s)
Total: 8.3s ⏰
Bounce rate: 73% 😱
Enter fullscreen mode Exit fullscreen mode

After Elasticsearch:

Search "red sneakers size 42"
└─ Single aggregated query (180ms)
Bounce rate: 12% 🎉
Conversion rate: +340% 📈
Enter fullscreen mode Exit fullscreen mode

Step-by-Step: Laravel + Elasticsearch Integration

🎯 Step 1: Install Laravel Scout + Elasticsearch Driver 

composer require laravel/scout
composer require matchish/laravel-scout-elasticsearch
Enter fullscreen mode Exit fullscreen mode

Add to your config/scout.php:

'driver' => env('SCOUT_DRIVER', 'elasticsearch'),

'elasticsearch' => [
    'index' => env('ELASTICSEARCH_INDEX', 'laravel'),
    'hosts' => [
        env('ELASTICSEARCH_HOST', 'http://localhost:9200'),
    ],
],
Enter fullscreen mode Exit fullscreen mode

🎯 Step 2: Configure Your Model 

use Laravel\Scout\Searchable;

class Product extends Model
{
    use Searchable;

    public function toSearchableArray()
    {
        return [
            'name' => $this->name,
            'description' => $this->description,
            'category_id' => $this->category_id,
            'brand' => $this->brand->name,
            'price' => $this->price,
            'status' => $this->status,
        ];
    }

    // 🔥 Here's where the magic happens
    public function searchableOptions()
    {
        return [
            'mappings' => [
                'properties' => [
                    'brand' => ['type' => 'keyword'],
                    'price' => ['type' => 'float'],
                    'category_id' => ['type' => 'integer'],
                ]
            ]
        ];
    }
}
Enter fullscreen mode Exit fullscreen mode

🎯 Step 3: Create Your Search Controller 

class ProductSearchController extends Controller
{
    public function search(Request $request)
    {
        $query = Product::search($request->query('q'));

        // Apply filters
        if ($category = $request->query('category')) {
            $query->where('category_id', $category);
        }

        // ✨ The part that replaces your foreach nightmare
        $results = $query->raw();

         $aggregations = [
            'categories' => $results['aggregations']['categories']['buckets'] ?? [],
            'brands' => $results['aggregations']['brands']['buckets'] ?? [],
            'price_ranges' => $results['aggregations']['price_ranges']['buckets'] ?? [],
        ];

        return response()->json([
            'products' => $query->paginate(20),
            'facets' => $aggregations
        ]);
    }
}
Enter fullscreen mode Exit fullscreen mode

🎯 Step 4: Index Your Data 

# Create the index
php artisan scout:import "App\Models\Product"

# Or do it in chunks for large datasets
php artisan tinker
>>> Product::chunk(100, fn($products) => $products->searchable());
Enter fullscreen mode Exit fullscreen mode

💡 Pro Tips That Saved My Production

Tip #1: Use Keyword Fields for Exact Matching 

// ❌ Wrong: Analyzed text field
'brand' => ['type' => 'text']

// ✅ Right: Keyword field for aggregations
'brand' => ['type' => 'keyword']
Enter fullscreen mode Exit fullscreen mode

Why? Text fields are analyzed and tokenized. "Red Bull" becomes ["red", "bull"]. Keyword fields stay intact, perfect for filters and aggregations.

Tip #2: Bulk Index for Large Datasets 

// Don't do this
Product::all()->searchable(); // 💀 RIP memory

// Do this
Product::chunk(100, function ($products) {
    $products->searchable();
}); // 😎 Smooth sailing
Enter fullscreen mode Exit fullscreen mode

Tip #3: Monitor Query Performance 

$response = $client->search($params);

Log::info('ES Query Time', [
    'took_ms' => $response['took'],
    'total_hits' => $response['hits']['total']['value']
]);
Enter fullscreen mode Exit fullscreen mode

Set up alerts if queries exceed 500ms. Something's wrong if ES is slow.

Tip #4: Use Query Cache Wisely 

'aggs' => [
    'categories' => [
        'terms' => [
            'field' => 'category_id',
            'size' => 100  // Limit buckets
        ]
    ]
]
Enter fullscreen mode Exit fullscreen mode

Don't request unlimited aggregation buckets. ES will cache the results, but be reasonable.

The "But What About..." Section

🤔 "Isn't Elasticsearch expensive?"

Not compared to over-provisioned MySQL instances. Plus, Elastic Cloud free tier exists. For most apps, a $45/month ES cluster beats a $200/month beefed-up RDS instance.

🤔 "My data changes constantly"

Elasticsearch handles near-real-time updates. Queue your index jobs with Laravel Jobs, problem solved. Updates appear in search within 1 second (configurable refresh interval).

🤔 "I only have 1000 products"

Then you probably don't need this. But when you scale to 100k, you'll thank yourself for setting this up early. Migration is painful when you're already on fire.

🤔 "Can't I just use Redis?"

Redis is great for caching, not for full-text search with aggregations. Different tools, different jobs. Redis = fast key-value store. Elasticsearch = search engine.

🤔 "What about Meilisearch or Algolia?"

  • Meilisearch: Great for simple search, lacks advanced aggregation features
  • Algolia: Amazing but $$$ for large datasets
  • Elasticsearch: Free, self-hosted, infinitely customizable

Choose based on your needs and budget.

Common Pitfalls (Learn From My Mistakes)

Mistake #1: Not Setting Field Types 

// This will haunt you
'brand' => ['type' => 'text']  // Can't aggregate efficiently

// Do this instead
'brand' => [
    'type' => 'text',
    'fields' => [
        'keyword' => ['type' => 'keyword']  // Use brand.keyword for aggs
    ]
]
Enter fullscreen mode Exit fullscreen mode

Mistake #2: Forgetting to Queue Index Jobs 

// This blocks your request
$product->save();
$product->searchable();  // Synchronous

// Do this
$product->save();
dispatch(new IndexProductJob($product));  // Async
Enter fullscreen mode Exit fullscreen mode

Mistake #3: Not Handling ES Downtime 

try {
    $results = Product::search($query)->get();
} catch (\Exception $e) {
    // Fallback to MySQL search
    Log::error('ES down, using fallback', ['error' => $e->getMessage()]);
    $results = Product::where('name', 'like', "%{$query}%")->get();
}
Enter fullscreen mode Exit fullscreen mode

Always have a fallback. ES is reliable, but stuff happens.

Performance Comparison Table

Metric MySQL (Manual) MySQL (Optimized) Elasticsearch
Query Time 8.3s 2.1s 0.18s
Concurrent Users 50 200 2000+
Code Complexity High Medium Low
Scalability Poor Fair Excellent
Aggregation Speed Slow Slow Fast
Full-Text Search Limited Limited Advanced

Before You Go: The Checklist

  • ✅ Identified slow search queries in your app
  • ✅ Installed Laravel Scout + Elasticsearch driver
  • ✅ Configured searchable models with proper mappings
  • ✅ Replaced manual aggregations with ES aggregations
  • ✅ Set up bulk indexing for existing data
  • ✅ Added monitoring for query performance
  • ✅ Implemented fallback for ES downtime
  • ✅ Set up async indexing with queues

Resources:

laravel #elasticsearch #php #webdev #backend #performance #searchoptimization

Top comments (0)