DEV Community

Cover image for From 400-Line Import Controllers to 20-Line Configs in Laravel
Robin
Robin

Posted on

From 400-Line Import Controllers to 20-Line Configs in Laravel

#webdev #php #laravel #opensource

The "Import Nightmares" We All Know

If you've built business applications with Laravel, you've definitely received this ticket:

"As an admin, I need to upload a CSV with 50,000 products to update our stock levels."

I've seen this request at least a dozen times across different projects. It sounds simple. So you grab a CSV parser, maybe league/csv or maatwebsite/excel, and start writing a Controller.

Ten minutes later, you're deep in the weeds:

  • "How do I validate row 49,000 without crashing memory?"
  • "The client calls the column 'E-Mail', but sometimes 'Email Address'."
  • "I need to find the Category ID by name, but create it if it doesn't exist."
  • "The client wants a 'Dry Run' to see errors before committing."

Your Controller becomes a 400-line monster of while loops, try-catch blocks, and manual validation logic. It's hard to test, hard to read, and terrifying to refactor.

There has to be a better way.

Introducing Laravel Ingest

I built Laravel Ingest to stop this madness.

Laravel Ingest is a configuration-driven framework for data imports. Instead of writing procedural scripts, you define what you want to import, and the package handles the how.

It takes care of the dirty work:

  • Streaming & Queues: Zero memory issues, whether 100 or 1 million rows.
  • Mapping & Transformation: Fluent API to map CSV columns to Eloquent attributes.
  • Relationships: Automatically resolves BelongsTo and BelongsToMany relations.
  • Dry Runs: Simulate imports to find errors without touching the database.
  • API & CLI: Auto-generated endpoints and Artisan commands.

Requirements

  • PHP 8.3+
  • Laravel 10, 11, or 12

How It Works

Let's say we want to import Users. Instead of a Controller, we create an Importer Class.

1. The Declarative Config

This is where the magic happens. Look how readable this is:

namespace App\Ingest;

use App\Models\User;
use LaravelIngest\Contracts\IngestDefinition;
use LaravelIngest\IngestConfig;
use LaravelIngest\Enums\SourceType;
use LaravelIngest\Enums\DuplicateStrategy;

class UserImporter implements IngestDefinition
{
    public function getConfig(): IngestConfig
    {
        return IngestConfig::for(User::class)
            ->fromSource(SourceType::UPLOAD)
            // Identify records by email
            ->keyedBy('email')
            // If email exists, update the record
            ->onDuplicate(DuplicateStrategy::UPDATE)
            // Map CSV 'Full Name' to DB 'name'
            ->map('Full Name', 'name')
            // Map CSV 'E-Mail' (or 'Email') to DB 'email'
            ->map(['E-Mail', 'Email'], 'email')
            // Transform 'yes/no' string to boolean
            ->mapAndTransform('Is Admin', 'is_admin', fn($val) => $val === 'yes')
            // Use Laravel Validation rules per row
            ->validate([
                'Full Name' => 'required|string|min:3',
                'E-Mail'    => 'required|email',
            ]);
    }
}
Enter fullscreen mode Exit fullscreen mode

2. Registering the Importer

In your AppServiceProvider, simply tag it:

$this->app->tag([UserImporter::class], 'ingest.definition');
Enter fullscreen mode Exit fullscreen mode

3. Running the Import

You don't need to write a Controller for uploads. Laravel Ingest automatically provides API endpoints and Artisan commands.

Via CLI (great for cronjobs or S3 imports):

php artisan ingest:run user-importer --file=users.csv
Enter fullscreen mode Exit fullscreen mode

Via API (for your frontend):

POST /api/v1/ingest/upload/user-importer
Body: file=@users.csv
Enter fullscreen mode Exit fullscreen mode

The Killer Features

1. Handling Relationships Is Finally Easy

Usually, importing related data (like assigning a Product to a Category by name) requires annoying lookup logic. Ingest does it in one line:

// Looks up the Category by 'name'. 
// If it doesn't exist, it creates it!
->relate(
    sourceColumn: 'Category Name',
    relation: 'category',
    relatedModel: Category::class,
    lookupColumn: 'name',
    createIfMissing: true
)
Enter fullscreen mode Exit fullscreen mode

2. Dry Runs Out of the Box

Clients hate it when an import fails halfway through. With Ingest, you can trigger a simulation that runs all validations and transformations but rolls back changes.

php artisan ingest:run user-importer --file=users.csv --dry-run
Enter fullscreen mode Exit fullscreen mode

3. Error Analysis API

When rows fail, you don't just get a log file. Ingest tracks failed rows in the database and provides an API endpoint to download a CSV of only the failed rows, including error messages. Your users can fix the errors in Excel and re-upload just those rows.

Under the Hood

Laravel Ingest stands on the shoulders of giants. It uses spatie/simple-excel to stream files line-by-line (keeping memory usage flat) and pushes chunks of rows onto the Laravel Queue.

This means your application stays responsive, even when importing a 500MB file.

Get Started

Installation is straightforward:

composer require zappzerapp/laravel-ingest
php artisan vendor:publish --provider="LaravelIngest\IngestServiceProvider"
php artisan migrate
Enter fullscreen mode Exit fullscreen mode

Resources:

Wrapping Up

I built this package because I was tired of writing the same fragile import code for every project. If you've ever dreaded the words "CSV upload", give Laravel Ingest a try.

Found it useful? Star the repo on GitHub, open an issue if you have feature requests, or drop a comment below. I'd love to hear how you're using it!

Top comments (0)