Effortless AWS S3 Integration in NestJS with @solegence/nest-js-aws-s3

Introduction

AWS S3 is a widely used cloud storage solution that provides secure, scalable, and high-performance object storage. Integrating S3 with NestJS applications can streamline file uploads, signed URL generation, and efficient data management. The @solegence/nest-js-aws-s3 package simplifies this integration, allowing developers to harness the power of S3 with minimal setup.

Why Use @solegence/nest-js-aws-s3?

This module provides:

• Easy AWS S3 integration
• Support for file uploads (PDF, CSV, and more)
• Chunked CSV uploads for handling large datasets
• Signed URL generation for secure access
• Bucket management (listing and deleting objects)
• TypeScript support for better development experience
• Configuration via environment variables

Installation

To get started, install the package:

npm install @solegence/nest-js-aws-s3

Configuration

Before using the module, set up your AWS credentials in a .env file:

AWS_S3_REGION=your-region
AWS_S3_BUCKET=your-bucket
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key

Registering the Module

You can register the S3Module in two ways:

• Synchronous Registration

import { S3Module } from '@solegence/nest-js-aws-s3';

@Module({
  imports: [
    S3Module.register({
      awsS3Region: process.env.AWS_S3_REGION,
      awsS3Bucket: process.env.AWS_S3_BUCKET,
      awsS3Accesskey: process.env.AWS_ACCESS_KEY_ID,
      awsS3SecretKey: process.env.AWS_SECRET_ACCESS_KEY,
      isGlobal: true,
    }),
  ],
})
export class AppModule {}

• Asynchronous Registration

import { S3Module } from '@solegence/nest-js-aws-s3';
import { ConfigService, ConfigModule } from '@nestjs/config';

@Module({
  imports: [
    S3Module.registerAsync({
      imports: [ConfigModule],
      useFactory: (configService: ConfigService) => ({
        awsS3Region: configService.get('AWS_S3_REGION'),
        awsS3Bucket: configService.get('AWS_S3_BUCKET'),
        awsS3Accesskey: configService.get('AWS_ACCESS_KEY_ID'),
        awsS3SecretKey: configService.get('AWS_SECRET_ACCESS_KEY'),
        isGlobal: true,
      }),
      inject: [ConfigService],
    }),
  ],
})
export class AppModule {}

Alternatively, you can use useClass or useExisting methods for dynamic configurations.

• Usage Examples

Uploading a Single File

async uploadSingleFile(filename: string, buffer: Buffer) {
  return await this.s3Service.uploadFile(this.AWS_S3_BUCKET, filename, buffer);
}

Uploading a PDF

@Injectable()
export class YourService {
  constructor(private readonly s3Service: S3Service) {}

  async uploadPdf(filename: string, buffer: Buffer) {
    try {
      const result = await this.s3Service.uploadFileInPdf(filename, buffer);
      return result;
    } catch (error) {
      throw new Error(`Failed to upload PDF: ${error.message}`);
    }
  }
}

Generating a Signed URL

async getFileUrl(key: string) {
  const url = await this.s3Service.getSignedUrl(key, 3600); // Expires in 1 hour
  return url;
}

Chunked CSV Upload

• For handling large datasets efficiently:

async uploadLargeCsv(key: string, data: any[]) {
  await this.s3Service.uploadCsvToS3InChunks(key, data);
}

Best Practices

• File Upload Recommendations
• Use uploadFileInPdf for PDFs and uploadfileInCsv for CSVs.
• Utilize uploadCsvToS3InChunks for large datasets.
• Optimize signed URL expiration times for security.

Error Handling

try {
  await s3Service.uploadFileInPdf('document.pdf', buffer);
} catch (error) {
  if (error.name === 'NoSuchBucket') {
    // Handle bucket not found
  } else if (error.$metadata?.httpStatusCode === 403) {
    // Handle permission issues
  }
}

Performance Optimization

• Use multipart uploads for files larger than 100MB.
• Implement compression for large files.
• Apply lifecycle policies for cost-effective storage management.

Troubleshooting

Common Issues & Solutions

• Access Denied

Verify AWS credentials and IAM permissions

Upload Failure

Check bucket name, file size, and network connectivity

Missing Config

Ensure environment variables are properly set

Contributing

We welcome contributions! To contribute:

Fork the repository.

Create a feature branch (git checkout -b feature/amazing-feature).

Commit your changes (git commit -m 'Add an amazing feature').

Push to the branch (git push origin feature/amazing-feature).

Open a Pull Request.

License

This project is licensed under the MIT License.

This module makes AWS S3 integration in NestJS projects seamless and efficient. Try it out and let us know your thoughts in the comments! 🚀