loading...

The strongest architecture to adopt in a startup.

yuno_miyako profile image ちぇん ・5 min read

Conclusion

aws (1)

  • ServerlessFramework is great
  • S3 hosting + CloudFront for the front end and Lambda@Edge for the SSR
  • Don't use RDS but DynamoDB as a database.
  • API is basically GraphQL, and you can easily add REST if you need it.

There is no absolute right answer to the architecture.

There is no absolute right answer for architecture.
This is because different products and phases have different requirements, and it is necessary to consider the appropriate architecture for them.

For example, the Tokyo Stock Exchange's high-speed trading system, which was recently closed for a day due to a failure, is an architecture that is fully allocated to stability and reliability requirements, distributing the load across 400 servers and localizing the impact of the failure by dividing the network.
Growth companies such as AWS have high scalability requirements and adopt a microservices architecture so that they can deploy new services without impacting the current application to keep innovating.

So let's consider what requirements are more important for startups.
Here is a ranking of the reasons why startups fail.

No. 1: there was no market need.
No. 2: Depletion of funds
No. 3: Inappropriate team composition
No. 4: Lost to the competition.
No. 5: Hard-to-use product

These are not all problems that can be solved by architecture, but you can see what requirements are needed.

No. 1: → Agility to change the specifications to meet customer feedback.
No. 2: → Cost optimization
No. 3: → Small team structure with as few specialists as possible
No. 4: → Increasing the speed of development to overtake the competition
No. 5: → Agility to increase the improvement cycle

We hypothesize that the requirements for startups are agility, cost optimization, small team, and speed of development.
We have examined the technical elements in this regard.

Technology Components

I've discussed how startups should adopt technology elements that excel in agility, cost optimization, small team, and speed of development.
I'm going to introduce the technology components that I think are superior in this regard.
I will explain in three stages: "front end", "database" and "back end (API)".

Frontend

Framework

There are three major front-end frameworks: React, Vue, and Angular.
In terms of "development speed" and "agility," I think typing in TypeScript is a must. The reason is that you can benefit from the completion and improve development efficiency by knowing what data to pass, and you can check for errors at compile time when you change the code due to specification changes.
In terms of compatibility with TypeScript, I think React is superior to the other two frameworks, and I think React should be adopted as a front-end framework.
Also, from the point of view of "speed of development" and "small team", I would recommend using Next.js rather than using bare React. This is because with Next.js, you can do things that are necessary but troublesome to implement with No Config. Specifically, routing and SSR (for SEO and SNS OGP).
From the above, Next.js (TypeScript) is recommended as a front-end framework.

Where to deploy the front end

The first places that come to mind for Next.js deployment are vercel, Netlify and Heroku.
These are good options because they are very easy to deploy and basically free to use.
However, there are always some functional limitations to the free quota, so if you're looking to scale, you'll want to build on AWS if possible.
With S3 hosting + CloudFront, you can use AWS free quota, and even if the free quota runs out, it's almost free (0.025 USD/GB for S3, 0.114 USD/10TB for CloudFront).
If you become a paid account for services such as vercel, you will have to pay a fixed cost, but with this configuration of AWS, it will be an on-demand billing, so we recommend S3 hosting + CloudFront in terms of "cost optimization".
By the way, since the free quota of vercel is also almost like unlimited, if you do not assume extreme scale, vercel is also totally good option.
I think that using a server such as EC2 or ECS as the deployment destination of the front end is out of the question from the point of view of "development speed" and "cost optimization".

Next.js SSR is done with Lambda@Edge.
Deploying this configuration is easy with ServerlessFramework with No Config.
https://github.com/serverless-nextjs/serverless-next.js

Database

RDS vs DynamoDB

In conclusion, I recommend you to choose DynamoDB without question.
To be more precise, RDS has to be used for complex data structures and access patterns, but in the initial phase, it is better to use DynamoDB and if it becomes complex, consider using RDS together.
From the point of view of cost optimization, Dynamo has a on-Demand option, while RDS charges by instance start-up time. Also, RDS+Lambda requires you to install RDS Proxy. There is also Aurora Serverless, but its cold start is critical.
Also, RDB has a dependency between tables, which makes it difficult to discard tables and causes migration and remaking of SQL.
In terms of development speed, Dynamo does not require management of database and is compatible with GraphQL, making it possible to increase development efficiency.

Designing DynamoDB

DynamoDB is recommended, but if you design DynamoDB like an RDB, you will get stuck, because DynamoDB doesn't allow joins, so you need to design your tables so that joins are not necessary.
RDB and NoSQL have completely different table design practices, so be sure to apply the initial learning curve here.

Backend(API)

REST vs GraphQL

We recommend GraphQL for agility, small team, and speed of development.
GraphQL transfers responsibility from the back end to the front end. It increases agility by not having to modify the API every time the UI is changed.
Also, if you write a REST API in Lambda, you will have to create as many Lambdas as there are endpoints (although there are ways not to do so), but GraphQL is a single endpoint, so you can keep it "small".
GraphQL is very easy to handle when using it from the front end, and although this is an experience, I feel that the front end development speed is higher than REST.

However, there are certain usecase for which REST is better suited. For example, file uploads require a little bit of work when using GraphQL. You can use REST endpoints in conjunction with an API Gateway in the same domain.

Deployment

Deploying the API Gateway + Lambda(GraphQL) + Lambda(REST) + DynamoDB configuration is easy with ServerlessFramework.
You can find templates for each language on here and try deploying it all the way through. You'll be surprised at how easy it is.
By the way, I use here after customizing it to make it easier to use.

About Certification Authorization

We use Cognito for authentication and authorization to implement login functions. If you are conscious of development speed, it may be faster to implement it yourself, but it's better to use a managed service because of the security implications.
This can also be added with ServerlessFramework.

Final Form.

aws (1)

In summary, the architecture is as shown in the above figure.

  • Using ServerlessFramework
  • S3 hosting + CloudFront for the front end and Lambda@Edge for the SSR
  • The database is DynamoDB, not RDS.
  • The API is basically GraphQL, you can easily add REST if needed.

I hope this will be helpful to those involved in startups.
Thank you.

Discussion

pic
Editor guide
Collapse
sharpninja profile image
The Sharp Ninja

I think you managed to pick the slowest combination of technologies to build an app! Plus, once you figure out relational data exists everywhere, that making one call to get relational data is faster than two (or more) to NoSQL, you will be re-writing your database. Next, you will discover that with extra load comes lots of calls to your microservices and that hosting them in an actual service is a LOT cheaper (and faster), so you'll be deploying new stuff to combine the services which will increase costs until your migration is done, at which point management will be fed up with this architecture and hire someone else to come redesign it from scratch and that person will make the same set of mistakes all over again.

Collapse
yuno_miyako profile image
ちぇん Author

Hmm, Designing data model of NoSQL to fetch desired data once can solve the problem you mentioned. And this is not micro service.

Collapse
sharpninja profile image
The Sharp Ninja

So you mean precalculate joins for all possible combinations of keys and then place each set in a document and stuff it away for possible reuse? That sounds very, very expensive.

Thread Thread
yuno_miyako profile image
ちぇん Author

You have to study how to model no sql tables. You don’t need it.
Actually RDB is more efficient storage usage but less efficient CPU usage.

Thread Thread
stewartjarod profile image
Jarod Stewart

Also should consider the adjacency pattern in dynamo. It's really powerful and almost completely removed the need of relational db

Thread Thread
sharpninja profile image
The Sharp Ninja

Maybe for relationships up to 2 levels deep, but real systems have hundreds of related tables, and often modules sold as add-ons add more tables that directly relate to core tables and a simple join is used to get the data. It is not uncommon for an app to need all invoices by a user that contain products with a special condition, such as being under manufacturer recall. Using proper 3BCNF normalization you would have a minimum of user -> invoice -> item -> special_attribute to get the result set to find users who ordered a product with a specific special attribute value. Are you really going to put a document in an Invoices catalog that contains mutable item attributes? If so, when the retailer was notified of the recall then every document containing that item would have to be updated, a very expensive operation eating up you quota of read and write operations. Then, you have the problem of eventual consistency where you cannot reliably perform the query to get the list of users until all the storage nodes in the redundancy set are updated. But unless you using data streaming and Lambda triggers, you cannot know when all the updates have propagated throughout the system, adding more cost and complexity.

Thread Thread
stewartjarod profile image
Jarod Stewart

The neat thing about dynamo is you can query many tables with cross table conditions with one requests as well... So that and adjacency pattern I have never come up to a problem where I was necessarily dying to have a relational store. I'm not saying dynamo is the perfect tool for all data types and relationships but it is very powerful and most people haven't used it to its full potential.

Thread Thread
yuno_miyako profile image
ちぇん Author

I totally agree with you. Most of people who say NoSQL is bad doesn't know basic practices of NoSQL.
However someone said that the instance cost of RDS is cheaper than the learning cost of NoSQL and I partially agree.

Collapse
storytellercz profile image
Jan Dvorak

Given what you have described in the reasons for starts-ups failure the setup still looks to me like a premature optimization. Also depends a lot on the product if this is going to work. Still if you already have some setup/framework ready to quickly deploy into your suggested setup then it might work.
This will depend a lot on the CTO. For example I would choose Meteor (with Apollo to mirror a bit your architecture) to get started quickly and remove the need to worry about wiring the infrastructure so that I can focus on the product and product-market fit.
I would also consider using MongoDB (with Atlas hosting) instead of DynamoDB, but I can see the value of keeping everything under one hosting provider.

よくやった

Collapse
yuno_miyako profile image
ちぇん Author

Thank you. I appreciate your insight. I'll study more!

Collapse
shenril profile image
Shenril

Thanks for the article !
I was wondering, any reason you didn t consider Aurora for RDB instead of Dynamo ? I guess it s a bit harder to adapt code to match the limitations of DynamoDB for regular APIs

Collapse
yuno_miyako profile image
ちぇん Author

Aurora is a part of RDS so I didn't mention the name. The reason why I didn't pick Aurora is same as mentioned in the article, cost(I don't want to pay fixed charge), agility(I don't want tables depended each other), development speed.
But as you say, DynamoDB has limitations and it would be critical for some requirements. We should consider requirements and design of data model.

Collapse
stewartjarod profile image
Jarod Stewart

I like this setup and it's generally what I prescribe as well. Throw in EventBridge and you are cooking