Stock Price Movement Reflections

Continuing on with improving my backend development skills, I decided to engineer a project using these new and unorthodox approaches to application development: Serverless & IaC. Here are some of my reflections regarding the experience and you can preview my project via: https://github.com/Jason0h/stock-price-movement

Part I: Serverless is Cool!

This project made use of a variety of AWS serverless services. Most notably, Python code was written for AWS Lambda. AWS Lambda is cool because as a Function as a Service, it removes the headache of having to learn and use a web framework such as Flask or Spring in order to write logic that serves requests, and then wrapping of all that in a container. With Lambda, you write plain Python code and deploy it.

When engineering my last project, I used the AWS console to provision my resources (in order words via many mouse clicks). But having learned about IaC coolness, I aimed to write a CloudFormation file that when deployed via the AWS SAM CLI would automatically provision the resources I needed. Frankly speaking, writing the configuration file was quite painful and took pretty long, missing the guiding information embedded in the console. However, the philosophy is that from now on, someone could create a copy of my infrastructure with a single key press. That being said, on future projects i'd probably provision resources via the AWS console first and then reverse engineer a CloudFormation file from that.

Part II: Thoughts on Writing Tests

Writing tests for code that is pure logic is pretty straightforward. Validate the functional output with the expected output. However, things are not as clear when what you're trying to validate is a side effect. For instance, in my Shorten URL project (for which I previously made a post about), I had to reason about how to test my database interaction logic. The key to the solution lied in a technique called "mocking", where I'd mock the actual database with an in memory database whose contents were cleared between tests, with the help of a testing framework (JUnit) that facilitated mocking.

I had to do something similar with this project, for instance with testing lambda logic that interacted with DynamoDB. Fortunately, AWS provides a Python library called moto for mocking AWS resources locally. With this I could run my lambda logic as many times I wanted locally without having to worry about disturbing my production database. It is clear to me that engineering application tests is an art in its own right, and ironically can sometimes be harder than writing the actual logic itself. It's definitely something that I will find worth leveling up on in my career.