Servers? What Are Those?

Ivan Wong — Tue, 20 Jul 2021 04:04:58 +0000

Last week, I wrote about how the first six weeks at skwirl went. If you haven't read that blog yet you can read it here and if this is the first time you've heard of skwirl you can learn more here. This week, I'll give an overview of the backend.

Kevin pitched skwirl to me about 4 months ago. Before he even finished his pitch I already had some ideas in mind for the tech stack. I wanted to stick with what I already knew to reduce time to market. I also wanted to use bleeding edge tech that I was fairly confident is going to become mainstream in the future. Lastly, I wanted to pick technology that was popular and actively being developed/maintained. My thought process at that point:

We'd start with a web app but we knew that we'd want a mobile app very soon
The front/backends should be separated
The backend would serve both the web frontend and a future mobile app
I spent several years developing Ruby on Rails apps before switching over to DevOps and working on AWS
Ideally, the backend would be some form of Ruby/ActiveRecord JSON API
I wanted to run everything on serverless (read to the end to learn why)
- This narrowed compute choices down to Lambda or Fargate
- Database options would be DynamoDB or Aurora
React was a strong contender for the front end
I have absolutely no front end experience

So I searched around the internet with the terms ruby lambda, ruby serverless, rails lambda, rails serverless. Eventually I came across and settled on using Ruby on Jets. Jets is actively being developed by Tung from BoltsOps. From the Jets homepage:

Ruby on Jets allows you to create and deploy serverless services with ease, and to seamlessly glue AWS services together with the most beautiful dynamic language: Ruby. It includes everything you need to build an API and deploy it to AWS Lambda. Jets leverages the power of Ruby to make serverless joyful for everyone.

What I like about Jets

Has the same project structure as Rails which makes it easy for those with Rails experience to learn
Deploys seamlessly to AWS with jets deploy as long as the machine you run the command on has sufficient permissions
- A minimum IAM policy for deployment is provided in the docs
Every controller action is a deployed as a separate lambda
- This makes it easy to rewrite specific requests in other languages if needed
Lambda function properties are fully customizable at the app, controller, or action level.
- This means request that need more memory/compute can be configured independently
- Access to AWS resources such as S3 can be scoped to just the requests that need them.
Prewarming is built in
routes.rb is automatically translated to API Gateway resources
- local server and console work the same as rails s and rails c
Scheduled jobs are automatically translated into Cloudwatch Event Rules
Fairly easy to write triggers for lambda. This along with the previous point eliminates the need for something like DelayedJob.
Ability to deploy extra copies of your app in the same environment
- This can be used in a CI/CD pipeline to deploy the next version of the app and test it against live data before the end users see the changes

How's performance?

Our staging environment is configured with 512MB of memory. It's deployed in us-east-1 (North Virginia) and I'm in Vancouver, BC. Currently, the request with the longest response time is /login. After being warmed up (we have prewarming turned off in staging), it takes ~950ms. Keep in mind that an expensive hashing function was chosen intentionally. A more reasonable gauge of performance would be /listings. This returns the first 20 active listings created by the current authenticated user. The average response time is ~130ms. Shave off 70ms or so for the Vancouver to North Virginia round trip and we're looking at ~40ms compute/DB time. Not the best but certainly good enough for our use cases especially considering we've done no optimization. We'll likely increase the function memory which increases CPU. Our database is also underpowered. Speaking of which...

The database

I tried very hard to work with DynamoDB but I just couldn't justify using under current conditions. After a month of part time learning (I was still employed at this time), I decided to go with Postgresql. My rationale:

One month of learning about DynamoDB/NoSQL can't compare to nearly a decade of experience working with Postgresql and ActiveRecord
Time to market >> cost and performance
If we ever get to the point where Postgresql doesn't scale enough, we have the best problem in the world (too many customers)
We can use a db.t2.micro (free tier) in staging and Aurora Serverless in production. Hopefully, this means we can set it and forget it in production. Ask me again in six months how this goes.
If we ever need anything like DynamoDB streams and triggering lambda off of DyanmoDB, we should be able to get similar results with Kinesis and Aurora
There are many Ruby Gems like Kaminari for pagination that works with Postgresql/ActiveRecord but not DynamoDB

Authentication

Any minor mistake implementing authentication can lead to catastrophic results. It's a solved problem and I'm not as smart as the people who already solved it. At this point, I already determined that we would eventually have multiple front ends (web and mobile). Session cookies were a no go. So I started looking for a solution that would manage auth/refresh tokens for me.

Cognito was the first one I tried. At first glace, it met all my requirements. Cognito would manage all usernames/passwords as well as JWTs. However, I soon realized that Cognito is a forever solution. There is no way to export password hashes out of Cognito. If we ever needed to migrate away from Cognito, we would have to force all users to reset their password.

In the end, I went with rodauth. Popular solutions like Devise wouldn't work since it's built for Rails. Rodauth on the other hand works with any rack based application. I was pleasantly surprised to discover such as feature rich authentication gem. Janko wrote a great article about it using it with Rails. If there's enough demand I might write an article about using it with Jets.

The Full Serverless Dream?

My serverless dream involves setting up the infrastructure once and forgetting about it. Hopefully, when we get some crazy Black Friday like traffic, Lambda and Aurora can work it's magic and scale without me lifting a finger. Will it work and be seamless for users? Ask me again in 6-12 months. We may not be completely serverless at that point though.

There's a high chance we'll need some sort of dedicated search service like Elasticsearch in the future. I'm not aware of any serverless solution AWS provides. AWS Elasticsearch Service and Cloudsearch are fully managed services. What's the difference? Fully managed services still require you to configure instance type/size, multi-AZ, and in some cases storage/backup. I'm fairly comfortable working with AWS services and would have no problem configuring them to be highly available and durable but if I can off load this work I will.

Next week I'll write about why I chose NextJS and TailwindCSS for the frontend.

I Went Nuts and Quit My Job to Start My Own Company

Ivan Wong — Tue, 13 Jul 2021 23:21:58 +0000

It's been six weeks since I left my job to start a company with my friend Kevin. We worked together at Galvanize. He was a technical product manager (called product designer at Galvanize) and I was the guy who crushed his dreams down into reality so I could actually build them (aka the developer). We have the same roles in our new company skwirl... or at least we're supposed to. Those of you who have read Kevin's blog may already know how things have deviated. Those of you who have started your own software companies probably already know that everyone has every role. So what's life been like so far?

I've never work so much in my life

I don't actually track my hours but if I had to guess, I'm probably working 80+ hours a week. It's 3AM right now and I'm not sure if I'm going to bed before the sun rises. I've pulled all-nighters several times already. But why am I working so much? Have I gone nuts? Is Kevin forcing me to work crazy hours? Nope, it's because...

I've never had so much fun in my life

Seriously! Working for ourselves and chasing our dreams is amazing. I've never felt so good in my life. Perhaps it's working with a someone I really trust and synergize with. Perhaps it's working in a domain I have a passion for. Perhaps it's taking taking control of our future. Perhaps this is just the honeymoon phase. It's probably all of the above and more.

Weird things I didn't see coming

I'm a CFO

It was pretty obvious that I would be in charge of integrating a payment processes (Paypal, Stripe, Square, etc) and managing that account. It turns out I'm also better with money than Kevin is. Next thing I know, I'm opening a business bank account.

Kevin is my UX dev intern

When I picked Next.js and TailwindCSS for the front end, I knew it was simple enough for Kevin to set up locally to tweak the UI. He is in charge of UX design until we get a real designer. My original plan was to build out the pages and components with most of the styling already done. I knew Kevin would want to tweak the styling once he had the pages rendered in his browser. He took it to the next level. The guy who was supposed to the technical product manager is making his own components and making them look good (with some supervision from me). This lead something unexpected...

The technical product manager suggested refactoring

What the f**k? In what world does a product manager (or any non-coder) suggest we refactor code? Before we even launch the product?! About two weeks in, our form components were getting messy and he actually said it.

Now you understand why devs want to refactor! But Kevin isn't the only one acting weird...

The developer (me) has suggested and is totally open to scope creep and moving the deadline earlier

Turns out when the product you're building is your (virtual) baby, scope creep ain't so bad. We've added some extra features that weren't part of our original plan and we're still somehow ahead of schedule. It might have something to do with the lack of meetings, customer requests/complaints, JIRA tickets to fill out, writing tests, and the unhealthy amount of red bull I've consumed.

What's coming?

If you're reading this blog post, chances are we've already launched our landing page. Over the next few weeks I'll be publishing some more blog posts going over the nuts and bolts of skwirl:

How can you help?

It's a two skwirl show here. We quit our jobs to chase our dream. If you are interested in what we are doing or share our vision there are a number of ways you can show your support. You can help us spread the word by sharing this post, or any other ones with your network and friends. You can support us on Buy Me a Coffee which unlocks different perks and rewards such as early access. You can come join the conversation on discord, facebook, twitter or wherever you feel most comfortable.

We cannot wait to share skwirl with the world. Until then, happy collecting 👋

DEV Community: Ivan Wong