DEV Community

Cover image for Laravel 13: Import a Large CSV File into the Database Without Running Out of Memory
Muhammad Yasir Amin
Muhammad Yasir Amin

Posted on Originally published at yasiramin.dev

Laravel 13: Import a Large CSV File into the Database Without Running Out of Memory

To import a large CSV file in Laravel 13 without running out of memory, read the file line by line with fgetcsv(), collect rows into batches of about 1,000, and write each batch with upsert(). Move the work into a queued job so the upload request finishes immediately.

What do you need before you start?

  • Laravel 13 on PHP 8.3 or newer
  • A products table with sku (unique), name and price columns
  • A CSV file with a header row: sku,name,price

Why does loading the whole file fail?

Functions such as file() or Excel::toArray() load every row into memory. A file with a few hundred thousand rows can exceed memory_limit. Streaming reads keep memory use flat, because only one batch is in memory at a time.

How do you prepare the table?

Schema::create('products', function (Blueprint $table) {
    $table->id();
    $table->string('sku')->unique();
    $table->string('name');
    $table->decimal('price', 10, 2);
    $table->timestamps();
});
Enter fullscreen mode Exit fullscreen mode

upsert() needs a unique index on the columns you pass as the "unique by" argument.

How do you write the importer?

<?php

namespace App\Services;

use App\Models\Product;
use RuntimeException;

class ProductCsvImporter
{
    public function __construct(private int $batchSize = 1000)
    {
    }

    public function import(string $path): int
    {
        $handle = fopen($path, 'r');

        if ($handle === false) {
            throw new RuntimeException("Cannot open {$path}");
        }

        $header = fgetcsv($handle);
        $batch = [];
        $total = 0;

        while (($row = fgetcsv($handle)) !== false) {
            if ($row === [null]) {
                continue; // blank line
            }

            $data = array_combine($header, $row);

            $batch[] = [
                'sku' => trim($data['sku']),
                'name' => trim($data['name']),
                'price' => (float) $data['price'],
                'created_at' => now(),
                'updated_at' => now(),
            ];

            if (count($batch) >= $this->batchSize) {
                $total += $this->flush($batch);
                $batch = [];
            }
        }

        fclose($handle);

        if ($batch !== []) {
            $total += $this->flush($batch);
        }

        return $total;
    }

    private function flush(array $batch): int
    {
        Product::upsert($batch, ['sku'], ['name', 'price', 'updated_at']);

        return count($batch);
    }
}
Enter fullscreen mode Exit fullscreen mode

upsert() inserts new rows and updates existing ones with the same sku in a single query per batch. It does not fire Eloquent model events.

How do you validate rows?

Validate each row cheaply before it enters the batch and count the failures instead of stopping the import.

use Illuminate\Support\Facades\Validator;

$validator = Validator::make($data, [
    'sku' => ['required', 'string', 'max:64'],
    'name' => ['required', 'string', 'max:255'],
    'price' => ['required', 'numeric', 'min:0'],
]);

if ($validator->fails()) {
    $skipped++;
    continue;
}
Enter fullscreen mode Exit fullscreen mode

How do you run it in a queue?

php artisan make:job ImportProductsCsv
Enter fullscreen mode Exit fullscreen mode
<?php

namespace App\Jobs;

use App\Services\ProductCsvImporter;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;

class ImportProductsCsv implements ShouldQueue
{
    use Queueable;

    public int $timeout = 900;

    public function __construct(public string $path)
    {
    }

    public function handle(ProductCsvImporter $importer): void
    {
        $importer->import(storage_path('app/'.$this->path));
    }
}
Enter fullscreen mode Exit fullscreen mode
$path = $request->file('csv')->store('imports');

ImportProductsCsv::dispatch($path);
Enter fullscreen mode Exit fullscreen mode

Run php artisan queue:work. Set the worker or job timeout longer than the expected import time.

How do you verify it works?

  1. Create a small CSV with a few rows and import it
  2. Run it again and confirm the row count does not grow, because upsert() updates by sku
  3. Import a large generated file and watch memory with memory_get_peak_usage(true)

What are the common mistakes?

  • Inserting one row at a time, which is slow for large files
  • Batches that are too large and exceed the database packet or placeholder limit
  • Forgetting the unique index required by upsert()
  • Non-UTF-8 files that need conversion before reading
  • Running the import in the web request and hitting the PHP time limit

FAQ

What batch size should I use?

Start with 500 to 1,000 rows and measure. The best size depends on column count and your database.

Can I use Laravel Excel instead?

Yes. Its chunk reading features can also handle large files, but the plain PHP approach has no extra dependency.

How do I report progress?

Store the total and processed counts in a table or cache key from the job and poll them from the UI.

References


Originally published on yasiramin.dev.

Top comments (0)