DEV Community

Avinash Dalvi for AWS Community Builders

Posted on • Updated on • Originally published at

Search nested subdirectory in S3 bucket

I am explaining about search nested subdirectory is exist in S3 bucket or not. Created AWS lambda code in python using boto3 to find existence of sub directory.

client.list_objects(Bucket=_BUCKET_NAME, Prefix=_PREFIX)

this function gives list of all content exist in bucket along with path. It will be easy to trace it out.

import boto3

client = boto3.client('s3')
bucket_name = "bucket_name"
prefix = ""

s3 = boto3.client("s3")

result = client.list_objects(Bucket=bucket_name, Delimiter='/')
   for obj in result.get('CommonPrefixes'):  
       prefix = obj.get('Prefix')
       file_list = ListFiles(client,bucket_name,prefix)
       for file in file_list:
          if "processed/files" in file:

def ListFiles(client, bucket_name, prefix):
    _BUCKET_NAME = bucket_name
    _PREFIX = prefix
    """List files in specific S3 URL"""
    response = client.list_objects(Bucket=_BUCKET_NAME, Prefix=_PREFIX)

    for content in response.get('Contents', []):
        yield content.get('Key')
Enter fullscreen mode Exit fullscreen mode

Github gist :

Stackoverflow :

Top comments (0)