Create a ‘batch’ app with AWS Lambda and Spring Boot and run it for FREE
I know, it sounds a bit crazy to use an AWS Lambda function for workloads that typically run in ‘batch’ mode, but there are some compelling reasons to do so. One reason is that, as a serverless technology, AWS Lambda lets you focus on implementing your workload without having to worry about the infrastructure on which it’s running. However, I think the most compelling reason reason is cost. If you have workloads that you can keep under the 15 minute limit of a Lambda function, or if you have a multi-step workload that can be broken into smaller (quicker) parts, you can set up one or more Lambda functions that run on a schedule. Depending on how often your workload needs to run, you may be able to get away with paying $0 for your compute resources!
It may also sound a bit crazy to run a Spring Boot app as a Lambda function. After all, Spring isn’t exactly a lightweight framework. I would probably not recommend setting up a Spring Boot enabled Lambda for public facing apps or APIs that require sub-second response times. However, for certain workflows where there’s no client waiting on a quick response, the setup I’m going to walk you through here can be pretty slick, and cheap. Did I mention cheap?
The application I’m going to walk you through is a pretty contrived example of something that an organization might actually need to do. I’m going to show you how to create a process that runs once per day to retrieve weather statistics from a public API and store that data in a database. The idea being that someone or some process will use that data at some point to run queries, do analytics, create data visualizations, or something else I haven’t thought of. Maybe in a future article, I’ll come up with some grandiose idea of what to do with all the data this application could gather over time. Let me know if you have any ideas!
My intention is to show you how to implement this application in Java, running as an AWS Lambda function that stores data in a MySQL RDS instance. In future articles, I plan to begin evolving parts of this implementation to show some different options like splitting this process into multiple Lambda functions, moving the RDS instance into a private VPC/subnet, and using S3 or DynamoDB to store the data. I also plan to explore using AWS Batch in place of the Lambda function. Eventually, I’d like to get to the point of showing you how to use some of the AWS data engineering services like AWS Glue, Athena, and others. There are so many AWS services that I could probably go on for years :)
Since there are a fair number of pieces that are going to be required to implement this system in a way that’s secure, maintainable, and testable, I’m not going to be able to cover them all in one article. So, I’m creating this article for you to use as as a jumping off point to explore each phase of the implementation. I’ll link each article from here so you’ll have access to all the details of creating this application. The GitHub repository for this little weather data collector app is here. Feel free to clone or fork the repository to follow along.
Here’s the list of detailed articles in the order that I think it makes sense to read/implement them. Articles that I’ve already written have links to them. The others are still in the works. So if you want to follow along with this little journey, bookmark this particular article because I’ll keep updating it with links to all the goodies.
- Configure Terraform to set up (and tear down) your MySQL RDS instance
- Configure Terraform to store your database credentials securely
- Configure AWS SAM to deploy your Lambda function and associated resources or ditch SAM and CloudFormation and deploy your Lambda function using Terraform
- Modify your Lambda function to connect to MySQL using credentials from AWS Secrets Manager
- Configure your Lambda function as a Spring Boot Application
- Implement best practice structured JSON logging
- Use Spring Data JPA to persist weather data to MySQL
Top comments (0)