Prelude
I've worked over several backend technologies, starting from PHP, then moving to RoR before getting my hands on NodeJS. I love how NodeJS simplifies backend development for Frontend developers; and not only this, the NPM ecosystem helps devs get up and running with complex projects with ease, without the need of re-inventing the wheel by developing core packages.
Over these years, I've tried and tested various folder structures for my projects (confession: I've never referred to any folder structure on the web as I wanted to design something of my own which I find comfortable working with rather than being biased upfront - how ironic as I am drafting this tutorial ๐) and I've finally reached a stage where I feel confident enough to share something which works for a majority of my NodeJS projects.
As an example for this article, we will try to structure an App assuming there's a:
- Website with a Landing Page
- Basic Authentication
- A Dashboard
- You are using Express like framework
- Build tools for Frontend like Gulp, Webpack.
- Applicable if you are using Frontend frameworks/libs like React, Angular, Vue.js, etc.
- Applicable if you wish to build API only App.
That's a pretty standard scenario for any application out there.
Last but not the least, the structure is more or less a Monorepo. You could take some good parts and apply them to your apps if your Node Apps are intended to serve as an API only or as a Website.
NodeJS Project Folder Structure
The folder structure I usually follow is inspired massively by RoR as I was working with Ruby & RoR before switching to NodeJS.
Here's how I structure my NodeJS App. I'll be explaining some of the reasoning behind the file structure and will also share some snippets on how do I expose some of the configs at a global level or how do I initiate a Database Connection across the app.
/app
โโโ config/
โ โโโ db.conf.js
โ โโโ app.conf.js
โ โโโ app.keys.js
โ โโโ db.keys.js
โ โโโ init.js
โโโ database/
โ โโโ Redis.database.js
โ โโโ Mongo.database.js
โ โโโ init.js
โโโ routes/
โ โโโ App.routes.js
โ โโโ Auth.routes.js
โ โโโ Dashboard.routes.js
โโโ utils/
โ โโโ Logger.util.js
โโโ middleware/
โ โโโ App.middleware.js
โ โโโ ErrorHandler.middleware.js
โ โโโ init.js
โโโ models/
โ โโโ User.model.js
โโโ controllers/
โ โโโ App.controller.js
โ โโโ User.controller.js
โโโ helpers/
โ โโโ App.helper.js
โโโ views/
โ โโโ layouts/
โ โโโ partials/
โ โโโ support/
โ โ โโโ index.ejs
โ โโโ documentation/
โ โ โโโ index.ejs
โ โโโ index.ejs
โ โโโ about.ejs
โ โโโ contact.ejs
/public
โโโ dist/
โโโ images/
โ โโโ dashboard/
โ โโโ auth/
โ โโโ documentation/
โโโ sitemap.xml
/samples
โโโ .env.sample
โโโ db.conf.sample
โโโ app.conf.sample
โโโ app.keys.sample
/src
โโโ javascript/
โโโ css/
/node_modules
/server.js
/package.json
/.env
Project Structure Brief
Config & Sample
Configuration could be categorised into three major categories:
- System Configuration
- App Configuration
- App Keys
Store the configuration variables in .env (dotenv) which are necessary for configuring your application like App Port, Environment (Production, Staging, etc.). .env configuration will be available across your app as they are set globally as ENV variables.
You could then create multiple app-relevant configuration files like:
- Database Connection: Database-specific configurations like Host, Port & Name.
- App Configuration: Acceptable Request Payload Size, Blacklisting IPs or Regions, etc.
- Auth Configuration: OAuth Callback URLs, etc.
And last but not the least, you could create multiple files and store relevant keys, like OAuth Client & Secret Keys, Database Keys, Mailer Keys etc.
Important: Do not commit any of these configuration files to Git. Copy the configuration structure with dummy values and commit to git using sample files under the
/samplefolder.
Database
This is where you could create connections to your database(s) like Redis, MySQL, MongoDB etc.
You could then require the necessary configuration & keys to initiate a connection here. This will also make it easier for you manage everything related to your Database connection in one file, like adding listeners around successful connections, errors and disconnects, etc.
Later, these connectors could be loaded under initialization files as needed.
Routes
Usually, you could create a single Route file here to manage all your app related routes but it is recommended to create multiple route files which could be categorised like:
- Website Routes
- API Routes
- Authentication Routes
- Documentation Routes
etc. Above is more scalable as well as manageable. Here's a code sample for the above:
const express = require("express");
...
const websiteRoutes = require("@routes/Website.routes");
const apiRoutes = require("@routes/Api.routes");
...
app.use("/", websiteRoutes);
app.use("/api", apiRoutes);
As you see, moving the routes to separate files also simplifies how your app would consume these routes.
Utils
Your app may have several utility functions, like Logging important information, or something which are static and have no relevance to other classes/files, as opposed to Helpers (defined later in this post) which may help other classes or modules in your app.
Middleware
Your app will have several middleware functions which could be separated and initiated in a single file. These helpers could range from critical middleware like Body Parser, Global Error Handlers, Authentication Middleware, enabling CORS, Attaching Custom Headers or setting a View Engine in ExpressJS.
Models
Every Collection (if MongoDB) or a Table (if MySQL) will have a standalone model file. For eg: A collection of Users will have it's own User.model.js file which could be extended further for defining a Schema Structure for the collection, Setting Default Values in the DB or Validating the User Inputs before storing the values to Database.
Controllers
Controllers will be responsible to handle all the incoming requests to your application which will either render a page in response, may send a JSON payload or will handle other critical API related actions like POST, PUT, DELETE etc.
Controllers will further consume Models, say a User model to Create Users while registering on your website or will render the view files if static pages are requested.
Helpers
Unlike utility methods, helpers could be dynamic in nature and related to specific controllers when needed. Helpers may contain methods to parse some user posted payload, modify it before storing it to the Database, etc.
Views
You may not need this folder if you are developing an API only app or using a separate SSR library like Next or Nuxt. This may be useful when you are using Express View Engine like Pug or EJS.
You can further divide your views into Layouts, Pages & Partials. Where, Layouts could be shared between similar pages and a website could have multiple layouts. Lastly, partials will hold common components like Header, Footer, Sidebar etc which are needed across your web pages.
Public
As the name suggests, anything under this folder will be publicly accessible by your website users. CSS, JavaScript and Images will be a part of this folder.
Your app may use tools like Webpack and Gulp which will compile(minify, pre-process) your (S)CSS & JS files and move under the public folder which will later be linked on web pages.
You could also build and output your React (or similar lib/framework) app under the dist/ folder.
Src
As App folder is responsible for handling all of the backend logic, Src folder will hold your JavaScript & CSS files if you are using SASS, ES 6 / Next which needs to be transpiled, compiled, minified and uglified before these could be served to the end users.
Src folder will further hold directories like CSS and JavaScript which could be customized as per your needs.
A simple explanation for init.js files before I close this article, init.js files will require rest of the files and export them to other files of your application, so that you need not have to require multiple files every time if you wish to consume them all.
Conclusion
I follow the above folder structure more or less in all the projects I build on Node. It works well for me and I would love to hear from you if it helps you in some way.
Eventually, you should try and modify the folder structure as per your needs and something which you find comfortable managing and scaling it in the long run.
Useful Resources
Lastly, I would love to share some packages which I use to manage configuration, simplifying the process of requiring files at various nested levels, as well as some Linting packages to standardise your code to a great extent..
Top comments (32)
I think it's not a secret for anyone that express.js is a dead project. Gets some packages updated every half of the year, and it hasn't changed in 5 years.
Outdated, and slow, with a whole bunch of legacy code supporting very old node versions. Why do you need It?
I highly recommend not making a monolith when your client side and server side are together in the same project. Create two projects and use the API.
I disagree that express is dead. Are you even up with the latest frameworks like NestJS, NextJS etc.. which have over a million downloads a week on npm registry? They all use express under the hood. Just because youโre not using it directly doesnโt mean it is dead. There are 65k dependent packages as we speak.
Lol Okay!
How is Expresssjs a dead project? They have been pushing out updates regularly.
Secondly I've stated that the above could be used as a website and as a standalone project if it's API only. The only thing you need to do is get rid of public and src like folders which are not relevant. Rest stays pretty much the same.
4.17.1 - May 26, 2019
4.17.2 - Dec 17, 2021
4.17.3 - Feb 17, 2022
4.18.0 - Apr 25, 2022
4.18.2 - 24 days ago
Incredibly regular updates for 4 years, you're right. 90% of new "features" with the prefix "deps". The remaining 10% are minor issues. But I do not dissuade you, use what is convenient for you. Some people still use jQuery and are quite happy with it.
What's wrong with jquery either? Not everything requires react or angular.
Express is pretty stable in its current state. I would rather see it as a framework which is matured enough that need not have to be updated as frequently as it doesn't break.
Look at the issues they have closed over the time, so what makes you think it isn't maintained? I am not against using something new or latest but a lot of devs these days want to use latest shiny frameworks without understanding the real use case.
Agree to your comment that we belong to different categories so our thoughts won't align so let's leave this discussion here.
Nice answer, without even reading the article.
What do you recommend in its place and why? How is it no longer doing the job that people use it for?
I find this very insightful. I don't really understand why people would keep suggesting and/or tell people to use a different library than the ones mentioned in this article, but I totally get that there are libraries that are better than the ones we use, but this focuses on the stuff that the majority of the developers/engineers use. I also do think this post is usually for those who want to learn and understand and gain different approaches and perspective from different people in terms of file architecture.
Stop using
dotenv. Use wj-config. It is far better.Not to be super critical, but you should probably disclose that you're the only contributor to wj-config, a project that's only a few months old, when you promote your own work. Telling someone to stop using something because your creation is "far better" as a single, isolated statement without justification, context, or evidence, is self-biased.
Yes, it is self-biased. I, however, would not invest time in trying to solve something that already has an acceptable solution. I come from the .Net environment where configuration is far more robust. When I had the need for configuration in the JavaScript realm it was a shock for me. I could not stand the current configuration solutions, so I made my own.
As far as being the only contributor or being a few months old, I don't see the relevance. The product can speak for itself, regardless of who made it, I would say.
It might not seem relevant to you as the author -- it's certainly relevant to everyone else in the community when you express authoritative (and biased, self-promoting) comments and tell somebody what to do (or stop doing), as you did. A bit of self-awareness as well as a little tact toward the author of the post you commented on is healthy, friend.
Ah, I see now what you meant. It is not about the capability of the package, it is about the social aspect of the sentence. I just wrote that quickly, as I do most comments, without a second thought.
I get it. I meant no disrespect towards the author. I simply volunteered an option without much second thought.
What problem does it solve in the above context as opposed to using dotENV?
It will allow you to consolidate your configuration into a single hierarchical calculated source as opposed to having to have multiple JS configuration files. It will support great flexibility regarding the source of your configuration, and on top of that, will keep your configuration much DRY'er.
Finally, as the cherry on top, if you are proxying or depending on other HTTP servers, wj-config will provide URL functions based on URL configuration data that can do route value replacement and dynamic query string addition. All replacements are URL encoded for you.
So, to answer your question, I'd say it solves problems you don't even know you have. Let me know if you are curious. I have a live demo in React @ CodeSandbox. Cheers.
Thank you for explaining. With due respect, no offence to your solution but I think dotENV would do a fair job here. I am aligned on what @subfuzion mentioned earlier, I had checked the details of the package before I asked you the question as well as your posts which were mostly on wj-config but I feel your package is not relevant to the article I've written.
That being said, I'll see if I could use it in some of the projects if needed :)
Thanks for the suggestion though.
No problem. To summarize on my intent, what I disliked due to my very own personal experience with
dotenvand your proposed folder structure is the need to have independent configuration sources. If you split the configuration, you split the maintenance too.Since
dotenvis not hierarchical in nature, it makes it very difficult and cumbersome to keep maintaining projects with it, of course, when compared to more powerful configuration approaches like .Net Configuration. Since I was able to bring the power of .Net Configuration to the JavaScript world, I figured I dropped the link here.If you were to use wj-config, you would quickly realize that your configuration folder would become simplified. That was the story behind the recommendation.
Use aex instead of express
What problem is aex solving that express doesnโt? What now the only thing I can see it is pushing the use of decorators, which feels more like something a Java insisted on.
Whatโs your relationship to aex?
Just take a look at:
github.com/calidion/aex
It has at lease the following advantages over express:
And 20% faster than express.
Are you the author of aex?
AEX is the only project that makes reference to โweb straight lineโ, from what I can see. Also articles on it on medium donโt have any feedback, making me nervous about something that doesnโt have a proper discussion.
What is wrong if I am the author? So you determine things by medium not logic?
There is nothing wrong if you are the author, just that you arenโt being transparent that you are. Itโs like recommending stocks, while not disclosing you are the owner of the business.
It helps us in deciding how objective an opinion is.
So you don't recognize thing by your brain only by listening to others?
Decorator is only a simple way to implement Web Straight Line.
Aex follows Web Straight Line instead of MVC which we think it is not suitable for web.
It was interesting to read your article and learn something new. I myself now work in a company and do database management. I recently had a problem, I didn't know how to quickly sync data in MySQL. So, I came across this article about sync MySQL databases Where I found a detailed guide on how to do everything. Perhaps someone will also find it useful, so I'll leave it here.
test
ะกongratulations ๐ฅณ! Your article hit the top posts for the week - dev.to/fruntend/top-10-posts-for-f...
Keep it up ๐