Hermann D. Schimpf

Posted on Sep 30 Originally published at hds-solutions.net

Where Does My Laravel App's Data Live in an Ephemeral Environment (Where Not Even the Disk Can Save You)

#serverless #laravel #dynamodb #s3

Versión en español aquí.

Spoiler: Lambda's disk exists. It's ephemeral, read-only, and self-destructs when AWS recycles the environment — and it doesn't tell you when.

In the previous chapter we turned Laravel into a digital nomad: we deployed to Lambda, served assets through CloudFront, and won the cold-start battle with Octane. But a promise was left hanging: where does your data end up when the disk is, literally, a temporary illusion?

Let's be clear: Lambda's disk exists. There's a writable /tmp directory where you can drop temporary files ¹. But treating it as real storage is like keeping your life savings in a shared taxi: technically they're in there, but good luck finding them tomorrow. The rest of the filesystem is read-only, and every Lambda instance can vanish at any moment, taking /tmp with it.

In this chapter we tackle persistence in three acts: first files (S3, the short part), then our entry into the NoSQL world with cache and sessions (DynamoDB, easy mode), and finally the main course: DynamoDB as the primary database.

1. The Serverless Treasure Map

Before we dive into the code, let's figure out where we're standing. This is the simplified architecture of my application:

In previous chapters we already covered how we use Lambda to run our Laravel app, and how CloudFront splits traffic between S3 (assets) and Lambda (the app). Today we'll:

Send uploads to S3.
Store session and cache data in DynamoDB.
And use DynamoDB as our primary database.

The other services are left for upcoming chapters. Patience, my young padawans.

2. Uploads to S3: The Short Part

This one is so simple it's almost embarrassing to dedicate a section to it. Almost.

a) The same old facade, a different destination

Laravel already abstracts storage with the Storage facade. On a traditional server it points to the local disk; in serverless, we simply define the bucket ² and switch the driver to s3 ³ ⁴:

composer require league/flysystem-aws-s3-v3

provider:
    environment:
        FILESYSTEM_DISK: s3
        AWS_BUCKET: ${construct:storage.bucketName}

constructs:
    storage:
        type: storage
        allowAcl: true
        # CORS is required to upload files directly from the browser using pre-signed URLs
        cors: ${construct:website.url}

And that's it. Every call to Storage::put(), Storage::get() now talks to S3 without touching a single line of business logic. Zero refactoring, pure abstraction, point for Laravel.

b) Pre-signed URLs: so uploads don't even go through Lambda

But here comes the clever trick. If the user uploads a file through your application, you're paying Lambda time for bytes that are just passing by. Remember the $530 bill for idle time from the previous article? Exactly.

The elegant alternative: pre-signed URLs ⁵. Laravel generates a temporary S3 URL, the frontend uses it to upload the file directly to S3, and Lambda never even finds out:

use Illuminate\Support\Facades\Storage;

// Laravel generates the URL, the browser uploads straight to S3
['url' => $url, 'headers' => $headers] = Storage::temporaryUploadUrl(
    "uploads/{$filename}", now()->addMinutes(5)
);

The file travels directly from the browser to S3 with no layovers. Generating the URL is a local signature, zero cost. Lambda keeps sleeping, and so does your wallet.

Lambda (and API Gateway) limit the payload to a maximum of 6 MB and 10 MB respectively ⁶. Beyond that size, pre-signed URLs stop being an optimization and become mandatory ⁷.

But if Lambda keeps sleeping... how does it find out a file was uploaded?

That's where the fun begins: S3 can automatically fire events ⁸ when a file is uploaded. That event can invoke a Lambda, broadcast a message through SNS, or emit an event to SQS or EventBridge. But we'll open that Pandora's box in another chapter.

3. DynamoDB, Easy Mode: Cache and Sessions

Before using DynamoDB as a database (the final boss of this article), let's start where it hurts the least: cache and sessions.

a) Cache: the table that cleans itself

The setup is insultingly simple ⁹. We only need to switch the cache store to dynamodb, and specify the DynamoDB table name:

provider:
    environment:
        CACHE_STORE: dynamodb
        DYNAMODB_CACHE_TABLE: !Ref CacheTable

And add the table definition in serverless.yml (inside resources, as CloudFormation) ¹⁰:

CacheTable:
    Type: AWS::DynamoDB::Table
    Properties:
        TableName: my-app-cache
        BillingMode: PAY_PER_REQUEST
        AttributeDefinitions:
            -   AttributeName: id
                AttributeType: S
        TimeToLiveSpecification:
            AttributeName: ttl
            Enabled: true
        KeySchema:
            -   AttributeName: id
                KeyType: HASH

The beautiful detail: TimeToLiveSpecification. DynamoDB automatically deletes expired items based on the ttl attribute ¹¹. No cleanup cron jobs, no cache:* cleanup commands, nothing. The trash takes itself out. DynamoDB's Roomba does the cleaning while you sleep.

Finally, we add the id and ttl attributes to the dynamodb configuration in config/cache.php ¹².

'dynamodb' => [
    'driver'     => 'dynamodb',
    'table'      => env('DYNAMODB_CACHE_TABLE', 'cache'),
    // ...
    'attributes' => [
        'key'        => 'id',
        'expiration' => 'ttl',
    ],
],

b) A new home for sessions

In the previous article we left sessions in cookies as a quick fix. It works, sure, but it has its limits: cookies travel with every request and are capped at ~4KB. For something more robust, Laravel ships with native DynamoDB support ¹³:

provider:
    environment:
        SESSION_DRIVER: dynamodb

Internally, Laravel reuses the same cache table ¹³ (with the same automatic TTL ¹¹). Sessions expire, DynamoDB sweeps them away, and you keep building features instead of maintaining infrastructure.

With cache and sessions migrated, only one stateful component is left standing: the database.

Grab some coffee, because it's about to get interesting.

4. DynamoDB as a Database: The Final Boss

This is where most Laravel devs get nervous. Our whole careers we were raised on CREATE TABLE, migrations, and JOINs. DynamoDB looks at all of that and says: "that's cute, but we don't do that here".

a) DynamoDB basics for Laravel devs

Let's translate the concepts into our language:

Table: same as always, but there are no migrations and no defined columns. Each item can have whatever attributes you want. Yes, whatever you want ¹⁴. Breathe.
Item: the equivalent of a row. Basically a JSON with its own identity.
PK (Partition Key): defines which physical partition the item lives in. It's the mandatory WHERE of every query.
SK (Sort Key): sorts items within the same partition. PK + SK form the composite primary key.
GSI (Global Secondary Index) ¹⁵: an alternative way to organize items so you can query them by other attributes. The closest thing to an index you'll find.

The golden rule that took me a while to internalize: in DynamoDB you don't model data, you model access patterns. First you think about which queries you'll run, then you design the keys. Unlike SQL, where you model normalized and then fight with the indexes.

b) Eloquent on DynamoDB: kitar/laravel-dynamodb

To simplify implementing Eloquent on top of DynamoDB I used the kitar/laravel-dynamodb package, which provides a connection driver and a query builder that translates Eloquent calls into DynamoDB operations.

composer require kitar/laravel-dynamodb

The connection in config/database.php looks like this:

'dynamodb' => [
    'driver'   => 'dynamodb',
    'key'      => env('AWS_ACCESS_KEY_ID'),
    'secret'   => env('AWS_SECRET_ACCESS_KEY'),
    'region'   => env('AWS_DEFAULT_REGION', 'us-east-1'),
    'token'    => env('AWS_SESSION_TOKEN'),
    'endpoint' => env('DYNAMODB_ENDPOINT'), // useful for local/testing
    'prefix'   => 'my-app-', // optional prefix for table names
],

provider:
    environment:
        DB_CONNECTION: dynamodb

And the models... are still Eloquent models, with a couple of extra attributes for the table keys ¹⁶:

use Kitar\Dynamodb\Model\Model;

class User extends Model
{
    protected $table          = 'users';
    protected $primaryKey     = 'email';    // partition key
    protected $sortKey        = 'type';     // sort key
    protected $sortKeyDefault = 'profile';
    protected $fillable       = ['name', 'email', 'password', 'type'];
}

sortKeyDefault covers the most common case: User::find('foo@bar.com') automatically uses 'profile' as the sort key. And if you need to query through a secondary index (for example, "all users" via a GSI):

User::index('GSI1')
    ->keyCondition('GSI1PK', '=', 'USER#')
    ->query();

You set the value of GSI1PK yourself when saving, it's just another attribute of the item.

See those weird keys? Values like USER# stuffed into a GSI1 index... that has a name of its own...

c) Single-Table Design: the concept (and nothing more)

Those weird keys are the gateway to Single-Table Design: instead of one table per entity (users, posts, comments...), several entities share a single table, told apart by their keys. The PK can identify the data's "owner" (USER#123), the SK what kind of thing it is (PROFILE, POST#456), and GSIs give you alternative views for other access patterns.

Sounds like black magic? It is. Properly designing a single-table layout is a whole art: it requires mapping all your access patterns before writing a single line of code, something you can improvise on the fly in SQL.

I won't go deeper here because the topic deserves its own article (and it will get one). Today you only need to know it exists, and that it's the reason DynamoDB can replace a full relational database.

d) When DynamoDB beats SQL, and when it doesn't

Let's be honest, no smoke and mirrors. DynamoDB is brutal when:

You know the access patterns upfront (get by id, list by owner, filter by status).
You need predictable latency: single-digit millisecond reads, at any scale ¹⁷.
Traffic is unpredictable: huge spikes or no traffic at all. PAY_PER_REQUEST: you pay per request, so with almost no traffic the bill is almost zero; with sustained high traffic, consider provisioned mode ¹⁸.
You want to scale without drama: there's no "master" node dying at 4 AM, no read replicas to configure to bypass max_connections. Scalability stops being your problem, it's now Jeff Bezos's problem.
DynamoDB Streams ¹⁹: every write can fire events. The equivalent of MySQL's binlog, but managed and capable of running code.
Zero management: no patching, no VACUUM, no scheduled maintenance.

And it's not for you if:

You need JOINs and ad-hoc queries: "show me all users who bought X last month grouped by state" → do it in SQL or export to a data warehouse.
Your data models change every week: redesigning keys in DynamoDB hurts more than a migration.
You depend on reporting/analytics: dashboards, aggregated charts, exploratory queries → you need SQL. DynamoDB is not the place.
You need to store giant items: DynamoDB has a maximum of 400KB per item ²⁰. Your documents go in S3, and only the reference goes in DynamoDB.
Scan as a strategy ²¹: you read the whole table and pay for all of it. If your query is solved with a scan, the design failed.

Access patterns like "posts by user", "comments by post", or direct reads by ID fit perfectly in the first group. Use cases tailor-made for DynamoDB.

e) Infrastructure is code too

Tables are declared as CloudFormation resources ²². The table for the example User model would look like this:

UsersTable:
    Type: AWS::DynamoDB::Table
    Properties:
        TableName: my-app-users
        BillingMode: PAY_PER_REQUEST
        AttributeDefinitions:
            - { AttributeName: email,  AttributeType: S }
            - { AttributeName: type,   AttributeType: S }
            - { AttributeName: GSI1PK, AttributeType: S }
            - { AttributeName: GSI1SK, AttributeType: S }
        KeySchema:
            - { AttributeName: email, KeyType: HASH }
            - { AttributeName: type,  KeyType: RANGE }
        GlobalSecondaryIndexes:
            -   IndexName: GSI1
                KeySchema:
                    - { AttributeName: GSI1PK, KeyType: HASH }
                    - { AttributeName: GSI1SK, KeyType: RANGE }
                Projection: { ProjectionType: ALL }

And the minimum IAM permissions in the main serverless.yml ²³:

provider:
    iam:
        role:
            statements:
                -   Effect: Allow
                    Resource:
                        - !GetAtt CacheTable.Arn
                        - !GetAtt UsersTable.Arn
                        - !Sub
                            - '${Resource}/index/*'
                            - { Resource: !GetAtt UsersTable.Arn }
                    Action:
                        - dynamodb:Query
                        - dynamodb:GetItem
                        - dynamodb:PutItem
                        - dynamodb:UpdateItem
                        - dynamodb:DeleteItem

Tip: We must never use Action: dynamodb:* or Resource: *. Least privilege or nothing ²⁴.

f) Costs: the last fixed bill goes away

Wrapping up the costs of this series:

From the previous setup: We were left with an RDS MySQL instance running. A db.t4g.small costs ~$26/month (≈ $0.032/h × 730 h + storage ²⁵) for a database that charges just for existing.
With DynamoDB: We can use BillingMode: PAY_PER_REQUEST and pay only for processed requests and stored GB. With modest traffic, the monthly bill is measured in cents: the database that charged for breathing is gone, and no component charges a flat fee for resources nobody is using anymore.

5. What's Next?

Today we figured out where everything persists: files in S3 (with uploads that never even touch Lambda), cache and sessions in DynamoDB with automatic expiration, and a NoSQL database that speaks Eloquent.

But the map still has unexplored areas. In the next article: queues. We'll see how SQS + a Lambda worker replace the queue:work that ran forever under a supervisor, and why that's better than praying to supervisord at 3 AM.

A question for you: have you already used DynamoDB with Laravel, or are you still on Team MySQL/PostgreSQL? Tell me in the comments what holds you back (or what made you fall in love) about the NoSQL world.

DEV Community