Hermann D. Schimpf

Posted on Sep 30 Edited on Oct 14 Originally published at hds-solutions.net

Where Does My Laravel App's Data Live in an Ephemeral Environment (Where Not Even the Disk Can Save You)

#serverless #laravel #dynamodb #s3

Versión en español aquí.

Spoiler: Lambda's disk exists. It's ephemeral, read-only, and self-destructs when AWS recycles the environment — and it doesn't tell you when.

In the previous chapter we turned Laravel into a digital nomad: we deployed to Lambda, served assets through CloudFront, and won the cold-start battle with Octane. But a promise was left hanging: where does your data end up when the disk is, literally, a temporary illusion?

Let's be clear: Lambda's disk exists. There's a writable /tmp directory where you can drop temporary files ¹. But treating it as real storage is like keeping your life savings in a shared taxi: technically they're in there, but good luck finding them tomorrow. The rest of the filesystem is read-only, and every Lambda instance can vanish at any moment, taking /tmp with it.

In this chapter we tackle persistence in three acts: first files (S3, the short part), then our entry into the NoSQL world with cache and sessions (DynamoDB, easy mode), and finally the main course: DynamoDB as the primary database.

1. The Serverless Treasure Map

Before we dive into the code, let's figure out where we're standing. This is the simplified architecture of my application:

In previous chapters we already covered how we use Lambda to run our Laravel app, and how CloudFront splits traffic between S3 (assets) and Lambda (the app). Today we'll:

Send uploads to S3.
Store session and cache data in DynamoDB.
And use DynamoDB as our primary database.

The other services are left for upcoming chapters. Patience, my young padawans.

2. Uploads to S3: The Short Part

This one is so simple it's almost embarrassing to dedicate a section to it. Almost.

a) The same old facade, a different destination

Laravel already abstracts storage with the Storage facade. On a traditional server it points to the local disk; in serverless, we simply define the bucket ² and switch the driver to s3 ³ ⁴:

composer require league/flysystem-aws-s3-v3

provider:
  environment:
    FILESYSTEM_DISK: s3
    AWS_BUCKET: ${construct:storage.bucketName}

constructs:
  storage:
    type: storage
    # Flysystem sends ACL headers on every operation and new buckets reject them by default [^12]
    allowAcl: true
    # CORS is required to upload files directly from the browser using pre-signed URLs
    cors: ${construct:website.url}

And that's it. Every call to Storage::put(), Storage::get() now talks to S3 without touching a single line of business logic. Zero refactoring, pure abstraction, point for Laravel.

b) Pre-signed URLs: so uploads don't even go through Lambda

But here comes the clever trick. If the user uploads a file through your application, you're paying Lambda time for bytes that are just passing by. Remember the $530 bill for idle time from the previous article? Exactly.

The elegant alternative: pre-signed URLs ⁵. Laravel generates a temporary S3 URL, the frontend uses it to upload the file directly to S3, and Lambda never even finds out:

use Illuminate\Support\Facades\Storage;

// Laravel generates the URL, the browser uploads straight to S3
['url' => $url, 'headers' => $headers] = Storage::temporaryUploadUrl(
    "uploads/{$filename}", now()->addMinutes(5)
);

The file travels directly from the browser to S3 with no layovers. Generating the URL is a local signature, zero cost. Lambda keeps sleeping, and so does your wallet.

Lambda (and API Gateway) limit the payload to a maximum of 6 MB and 10 MB respectively ⁶. Beyond that size, pre-signed URLs stop being an optimization and become mandatory ⁷.

But if Lambda keeps sleeping... how does it find out a file was uploaded?

That's where the fun begins: S3 can automatically fire events ⁸ when a file is uploaded. That event can invoke a Lambda, broadcast a message through SNS, or emit an event to SQS or EventBridge. But we'll open that Pandora's box in another chapter.

3. DynamoDB, Easy Mode: Cache and Sessions

Before using DynamoDB as a database (the final boss of this article), let's start where it hurts the least: cache and sessions.

a) Cache: the table that cleans itself

The setup is insultingly simple ⁹. We only need to switch the cache store to dynamodb, and specify the DynamoDB table name:

provider:
  environment:
    CACHE_STORE: dynamodb
    DYNAMODB_CACHE_TABLE: !Ref CacheTable

And add the table definition in serverless.yml (inside resources, as CloudFormation) ¹⁰:

CacheTable:
  Type: AWS::DynamoDB::Table
  Properties:
    TableName: my-app-cache
    BillingMode: PAY_PER_REQUEST
    AttributeDefinitions:
      - AttributeName: key
        AttributeType: S
    TimeToLiveSpecification:
      AttributeName: expires_at
      Enabled: true
    KeySchema:
      - AttributeName: key
        KeyType: HASH

The beautiful detail: TimeToLiveSpecification. DynamoDB automatically deletes expired items based on the expires_at attribute ¹¹. No cleanup cron jobs, no cache:* commands, nothing. DynamoDB's Roomba does the cleaning while you sleep.

The key and expires_at attributes are the default names Laravel uses ⁹.

b) A new home for sessions

In the previous article we left sessions in cookies as a quick fix. It works, sure, but it has its limits: cookies travel with every request and are capped at ~4KB. For something more robust, Laravel ships with native DynamoDB support ¹²:

provider:
  environment:
    SESSION_DRIVER: dynamodb

Internally, Laravel reuses the same cache table ¹² (with the same automatic TTL ¹¹). Sessions expire, DynamoDB sweeps them away, and you keep building features instead of maintaining infrastructure.

The honest trade-off: cookies were free; now every authenticated request is one read and one write in DynamoDB. With PAY_PER_REQUEST and modest traffic we're talking cents per month, but it's worth knowing before it shows up on the bill.

With cache and sessions migrated, only one stateful component is left standing: the database.

Grab some coffee, because it's about to get interesting.

4. DynamoDB as a Database: The Final Boss

This is where most Laravel devs get nervous. Our whole careers we were raised on CREATE TABLE, migrations, and JOINs. DynamoDB looks at all of that and says: "that's cute, but we don't do that here".

a) DynamoDB basics for Laravel devs

Let's translate the concepts into our language:

Table: same as always, but with no migrations and no columns: each item can have whatever attributes it wants. The only exception is the attributes you use as keys or indexes ¹³: those you do have to declare.
Item: the equivalent of a row. Basically a JSON with its own identity.
PK (Partition Key): defines which physical partition the item lives in. It's the mandatory WHERE of every query. The only one that gets a pass is Scan, but in section 4.d you'll see why you won't want to use it.
SK (Sort Key): sorts items within the same partition. PK + SK form the composite primary key.
GSI (Global Secondary Index) ¹⁴: an alternative way to organize items so you can query them by other attributes. The closest thing to an index you'll find.

The golden rule that took me a while to internalize: in DynamoDB you don't model data, you model access patterns. First you think about which queries you'll run, then you design the keys. Unlike SQL, where you model normalized and then fight with the indexes.

b) Eloquent on DynamoDB: kitar/laravel-dynamodb

To simplify implementing Eloquent on top of DynamoDB I used the kitar/laravel-dynamodb package, which provides a connection driver and a query builder that translates Eloquent calls into DynamoDB operations.

composer require kitar/laravel-dynamodb

The connection in config/database.php looks like this:

'dynamodb' => [
    'driver'   => 'dynamodb',
    'key'      => env('AWS_ACCESS_KEY_ID'),
    'secret'   => env('AWS_SECRET_ACCESS_KEY'),
    'region'   => env('AWS_DEFAULT_REGION', 'us-east-1'),
    'token'    => env('AWS_SESSION_TOKEN'),
    'endpoint' => env('DYNAMODB_ENDPOINT'), // useful for local/testing
    'prefix'   => 'my-app-', // optional prefix for table names
],

The values of AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN are automatically injected into the environment from the function's execution role. They're temporary credentials that rotate on their own, and the permissions they carry are exactly the ones we declare in IAM (section 4.e).

provider:
  environment:
    DB_CONNECTION: dynamodb

And the models... are still Eloquent models, with a couple of extra attributes for the table keys ¹⁵, and a bit of glue for the GSI:

use Illuminate\Database\Eloquent\Casts\Attribute;
use Illuminate\Support\Str;
use Kitar\Dynamodb\Model\Model;

class User extends Model
{
    protected $table          = 'users';
    protected $primaryKey     = 'PK';     // partition key → USER#<uuid>
    protected $sortKey        = 'SK';     // sort key
    protected $sortKeyDefault = 'PROFILE';
    protected $fillable       = ['name', 'email', 'password'];
    protected $appends        = ['email'];  // so it shows up when serializing

    protected static function booted(): void
    {
        static::creating(function (self $user) {
            $user->PK       = 'USER#'.Str::uuid();
            $user->GSI1PK   = 'USER'; // groups all users together
        });
    }

    protected function email(): Attribute
    {
        // the email lives in GSI1SK: the same index serves both listing users and looking up by email
        return Attribute::make(
            get: fn (): string => $this->GSI1SK,
            set: static fn (string $email): array => [ 'GSI1SK' => $email ],
        );
    }
}

Careful with the "still Eloquent" part: they inherit mutators, casts, events, serialization and authentication, but not relationships (hasMany(), belongsTo(), with()), nor a generic where() (here it's keyCondition(), filter() or condition()), nor arbitrary orderBy() (only by Sort Key), nor page-number paginate() (DynamoDB paginates by cursor), nor count()/aggregates. It's not a package limitation: DynamoDB simply doesn't know how to do that, and the package is honest enough not to pretend otherwise.

Why a UUID and not the email as the Partition Key, which is what the package's README does? Because in DynamoDB the primary key is the item's identity: it can't be modified, only deleted and recreated. The day a user wants to change their email, you'd have to recreate the item and everything hanging off that PK. An immutable identifier as PK and the email in a GSI lets you look up by email today and change it tomorrow without touching the item's identity.

sortKeyDefault covers the most common case: User::find('USER#2aab…11a2') automatically uses PROFILE as the sort key. And the GSI solves the other two access patterns:

// all users
User::index('GSI1')->keyCondition('GSI1PK', '=', 'USER')->query();

// the user with this email
User::index('GSI1')
    ->keyCondition('GSI1PK', '=', 'USER')
    ->keyCondition('GSI1SK', '=', 'foo@bar.com')
    ->query()->first();

A detail that bites: the GSI lets you look up by email, but it does not guarantee it's unique. Two users with the same email would happily coexist in the index. If you need uniqueness, you validate it before creating.

See those weird keys? USER#<uuid> as the primary key, USER stuffed into a GSI1 index... that has a name of its own...

c) Single-Table Design: the concept (and nothing more)

Those weird keys are the gateway to Single-Table Design: instead of one table per entity (users, posts, comments...), several entities share a single table, told apart by their keys. The PK can identify the data's "owner" (USER#123), the SK what kind of thing it is (PROFILE, POST#456), and GSIs give you alternative views for other access patterns.

Sounds like black magic? It isn't, but it is an art: properly designing a single-table layout requires mapping all your access patterns before writing a single line of code, something you can improvise on the fly in SQL.

I won't go deeper here because the topic deserves its own article (and it will get one). Today you only need to know it exists, and that it's the reason DynamoDB can replace a full relational database.

d) When DynamoDB beats SQL, and when it doesn't

Let's be honest, no smoke and mirrors. DynamoDB is brutal when:

You know the access patterns upfront (get by id, list by owner, filter by status).
You need predictable latency: single-digit millisecond reads, at any scale ¹⁶.
Traffic is unpredictable: huge spikes or no traffic at all. PAY_PER_REQUEST: you pay per request, so with almost no traffic the bill is almost zero; with sustained high traffic, consider provisioned mode ¹⁷.
You want to scale without drama: there's no "master" node dying at 4 AM, no read replicas to configure to bypass max_connections. Scalability stops being your problem.
DynamoDB Streams ¹⁸: every write can fire events. The equivalent of MySQL's binlog, but managed and capable of running code.
Zero management: no patching, no VACUUM, no scheduled maintenance.

And it's not for you if:

You need to sort or filter by any attribute: in DynamoDB you only sort by Sort Key and only search efficiently by keys (or GSIs). Everything else is filter() over what's already been read: you pay to read and then discard.
You need JOINs and ad-hoc queries: "show me all users who bought X last month grouped by state" → do it in SQL or export to a data warehouse.
You depend on UNIQUE, foreign keys or ON DELETE CASCADE: they don't exist. Uniqueness is enforced by hand with condition expressions (attribute_not_exists(PK)), and referential integrity is your code's responsibility, not the database's.
Your data models change every week: redesigning keys in DynamoDB hurts more than a migration.
You depend on reporting/analytics: dashboards, aggregated charts, exploratory queries → you need SQL. DynamoDB is not the place.
You need to store giant items: DynamoDB has a maximum of 400KB per item ¹⁹. Large documents go in S3, and only the reference goes in DynamoDB.
Scan as a strategy ²⁰: you read the whole table and pay for all of it. If your query is solved with a scan, the design failed.

Access patterns like "posts by user", "comments by post", or direct reads by ID fit perfectly in the first group. Use cases tailor-made for DynamoDB.

e) Infrastructure is code too

Tables are declared as CloudFormation resources ²¹. The table for the example User model would look like this:

UsersTable:
  Type: AWS::DynamoDB::Table
  Properties:
    TableName: my-app-users
    BillingMode: PAY_PER_REQUEST
    AttributeDefinitions:
      - { AttributeName: PK,     AttributeType: S }
      - { AttributeName: SK,     AttributeType: S }
      - { AttributeName: GSI1PK, AttributeType: S }
      - { AttributeName: GSI1SK, AttributeType: S }
    KeySchema:
      - { AttributeName: PK, KeyType: HASH }
      - { AttributeName: SK, KeyType: RANGE }
    GlobalSecondaryIndexes:
      - IndexName: GSI1
        KeySchema:
          - { AttributeName: GSI1PK, KeyType: HASH }
          - { AttributeName: GSI1SK, KeyType: RANGE }
        Projection: { ProjectionType: ALL }

And the minimum IAM permissions in the main serverless.yml ²²:

provider:
  iam:
    role:
      statements:
        - Effect: Allow
          Resource:
            - !GetAtt CacheTable.Arn
            - !GetAtt UsersTable.Arn
            - !Sub
              - '${Resource}/index/*'
              - { Resource: !GetAtt UsersTable.Arn }
          Action:
            - dynamodb:Query
            - dynamodb:GetItem
            - dynamodb:PutItem
            - dynamodb:UpdateItem
            - dynamodb:DeleteItem

Tip: We must never use Action: dynamodb:* or Resource: *. Least privilege or nothing ²³.

f) Costs: the last fixed bill goes away

Let's update the tally:

The debt from the first chapter: Back there we left an RDS MySQL db.t4g.small running, which costs ~$26/month (≈ $0.032/h × 730 h + storage ²⁴) for a database that charges just for existing. Today we retire it.
With DynamoDB: We can use BillingMode: PAY_PER_REQUEST and pay only for processed requests and stored GB. With modest traffic, the monthly bill is measured in cents: the database that charged for breathing is gone, and no major component charges a flat fee for resources nobody is using anymore.

5. What's Next?

Today we figured out where everything persists: files in S3 (with uploads that never even touch Lambda), cache and sessions in DynamoDB with automatic expiration, and a NoSQL database that speaks Eloquent.

But the map still has unexplored areas. In the next article: queues. We'll see how SQS + a Lambda worker replace the queue:work that ran forever under a supervisor, and why that's better than praying to supervisord at 3 AM.

A question for you: have you already used DynamoDB with Laravel, or are you still on Team MySQL/PostgreSQL? Tell me in the comments what holds you back (or what made you fall in love) about the NoSQL world.

DEV Community