Day 46: Compose Wires the Whole Stack, and Permission to Invoke Is Not Permission to Act

#aws #docker #devops #serverless

Two things today needed wiring on both sides, and in both cases getting one side right produces something that looks finished and does nothing.

One Docker task, one AWS task. Stand up a PHP and MariaDB stack with Compose, then build an event pipeline where an S3 upload triggers a Lambda that copies the object and logs it. The tasks come from the KodeKloud Engineer platform.

Two services, one command

services:
  web:
    image: php:7.2-apache
    container_name: php_host
    ports:
      - "8085:80"
    volumes:
      - /var/www/html:/var/www/html

  db:
    image: mariadb:latest
    container_name: mysql_host
    ports:
      - "3306:3306"
    volumes:
      - /var/lib/mysql:/var/lib/mysql
    environment:
      MYSQL_DATABASE: database_host
      MYSQL_USER: <username>
      MYSQL_PASSWORD: "<password>"
      MYSQL_RANDOM_ROOT_PASSWORD: "yes"

sudo docker compose -f /opt/sysops/docker-compose.yml up -d
curl http://localhost:8085

One up -d creates a project network, both containers and both mounts. The two containers can reach each other by service name over that network, which is the Day 42 embedded DNS point finally doing the job it exists for. Nothing here declares that relationship; it comes free with the project network.

Worth separating two names that look interchangeable and are not. The db service is addressable from web as db, the service key. container_name only fixes what docker ps and docker exec see. Pointing a connection string at mysql_host works because the container name resolves too, but db is the portable one.

Note also that this mount targets /var/www/html while yesterday's httpd file wanted /usr/local/apache2/htdocs. Same idea, different image convention, and it is worth reading the image's documentation rather than assuming a path you have seen before.

The one that catches people: those MariaDB environment variables are the image's initialisation contract, and they only apply on first start. The image initialises the database only when the data directory is empty. Mount a /var/lib/mysql that already has data and every one of those variables is ignored, in silence. A stack that "will not pick up the new password" is nearly always this.

And publishing 3306 is what the task asked for, not what you would otherwise do. The web container reaches the database over the project network with no published port at all. Putting 3306 on the host exposes the database to anything that can reach the host.

Every permission it needed, and never called

The AWS task was six resources in one path: an upload to a public bucket fires an S3 event, Lambda copies the object into a private bucket and writes an audit row to DynamoDB.

The execution role covered s3:GetObject, s3:PutObject and dynamodb:PutItem, scoped to exactly one bucket each and one table. Correct, least-privilege, and completely insufficient, because none of it lets S3 call the function.

Policy	Attached to	Answers
Identity-based	The role	What is Lambda allowed to do?
Resource-based	The function	Who is allowed to invoke Lambda?

aws lambda add-permission --function-name devops-copyfunction \
  --statement-id s3invoke --action lambda:InvokeFunction \
  --principal s3.amazonaws.com \
  --source-arn arn:aws:s3:::$PUB_BUCKET --source-account $ACCT

Get only the identity side right and nothing errors. There are no logs, because the function never ran. This is the same shape as trust policy versus permissions policy on Day 33 and execution role versus task role on Day 38, and it keeps arriving in a new costume.

The ordering is load-bearing too. add-permission has to come before put-bucket-notification-configuration, because S3 validates the destination when you set a notification. Without the permission you get:

Unable to validate the following destination configurations

Which mentions neither Lambda nor permissions nor what to change. Same category as AccessDenied on PutBucketPolicy actually meaning BlockPublicPolicy: the error names the call that failed rather than the setting that caused it. The console hides this by issuing both calls when you add a trigger through the UI.

Scope the permission, or anyone can use it

Lambda's resource-based permissions support a deliberately small set of condition keys: aws:SourceArn, aws:SourceAccount and aws:PrincipalOrgID. Leave the first two off and the statement allows any S3 bucket, in any account, to invoke your function.

That is the confused deputy problem in one line. Someone creates a bucket, points a notification at your function ARN, and your code runs on their objects with your permissions. The condition block you want to see in the response:

"Condition": {
  "StringEquals": {"AWS:SourceAccount": "245695940513"},
  "ArnLike": {"AWS:SourceArn": "arn:aws:s3:::devops-public-22976"}
}

Three details worth stealing

128 MB handles any file size here, and the reason is not generosity. s3.copy_object is server-side: S3 copies the object internally and the bytes never pass through the Lambda execution environment. Had the code done get_object then put_object instead, the file would flow through Lambda memory and 128 MB would cap the file size hard. Same apparent outcome, completely different resource profile.

put-bucket-notification-configuration takes the whole configuration document. There is no append. On a bucket already sending events to SQS or SNS, a straight write removes them silently, so the safe pattern on anything you did not create is read, merge, write.

And a hyphen is not valid in a Python module name. The provided file is lambda-function.py, the handler string is module.function, and lambda-function is not importable. Renaming it during the build step is a one-character fix that otherwise surfaces as an import error at invoke time, minutes after everything else looked fine.

The bug in the error handler

Worth recording, because it is a realistic failure mode rather than a criticism of the lab:

try:
    source_bucket = event['Records'][0]['s3']['bucket']['name']
    ...
except Exception as e:
    log_entry = {
        'SourceBucket': source_bucket,      # unbound if the first line threw

If that first line raises, on a malformed or non-S3 event, source_bucket was never assigned. The except block then throws NameError while building the error log, and the original exception is lost.

A handler written to record failures, unable to record the one class of failure that happens before its variables exist. Initialising them to None above the try fixes it. Not needed for the task to pass, and exactly how an error handler ends up hiding the error it was written to surface.

Both sides, or nothing moves

Compose wires two containers into one project, and the relationship is implicit. Lambda needs two policies pointing in opposite directions, and neither implies the other.

The tell in both cases is the same: a component that is individually correct and collectively inert. A db container running perfectly that web cannot name. A function with exactly the right permissions that nothing is allowed to call.

So here is the Day 46 question. When you grant access to something, do you check both halves, what it can do and who can reach it, or only the half that was in the ticket?

Day 46 down. Fifty-four to go.