Development Experience of the User Management Project with NestJS

Introduction

Developing modern web applications requires a comprehensive and robust approach to user management. This article presents the experience and knowledge gained during the development of a user management microservice using NestJS, a progressive framework for Node.js that helps build efficient and scalable server-side applications. The complete source code and documentation can be found on the GitHub repository.

The user management project addresses multiple critical aspects for any web application, from user registration and authentication to password recovery and token validation. Implementing these functionalities not only ensures a better user experience but also reinforces the security and integrity of the application.

In this article, I will detail the prior and acquired knowledge during the project development, the technical and managerial challenges faced, the tools and technologies used, and the lessons learned. Key aspects of the development process, including agile methodologies, continuous integration, and continuous deployment (CI/CD), will also be covered.

Purpose of the Article

The purpose of this article is to provide a detailed and practical overview of developing a user management system. This journey will include everything from theoretical knowledge to technical implementation and production deployment in a cloud environment using Kubernetes. Additionally, personal experiences of facing and overcoming challenges, both technical and managerial, will be shared, offering valuable insights for other developers and development teams embarking on similar projects.

Brief Project Description

The user management project focuses on creating a microservice that handles all tasks related to user administration. Using NestJS as the main framework, the project includes essential functionalities such as new user registration, authentication through JWT token generation, password recovery via email, and token validation to ensure secure access to application resources.

The application was deployed on Google Kubernetes Engine (GKE), leveraging Kubernetes' orchestration and scalability capabilities. This environment allowed for efficient management of the underlying infrastructure and ensured that the user management service was always available and functioning optimally.

Context and Motivation

The motivation behind this project arises from the need for a reliable and secure user management system that can easily integrate into any web application. User management is a critical functionality that, if not implemented correctly, can compromise both security and the end-user experience. By developing this microservice, the goal was not only to meet these requirements but also to explore and apply modern technologies and best practices in software development and DevOps.

This project was not only an opportunity to apply existing knowledge but also to learn and master new tools and technologies essential in today's web development landscape. Throughout this article, a comprehensive overview of the process will be provided, from project conception to implementation and final deployment, highlighting lessons learned and best practices adopted.

Prior and Acquired Knowledge

Developing the user management project using NestJS provided a significant opportunity to apply previous knowledge and acquire new technical and conceptual skills. The following outlines the prior knowledge and the skills acquired during the project's development.

Prior Knowledge

Before starting the project, I had a solid foundation in several fundamental aspects of software development and cloud services:

• Cloud Services: I had basic knowledge about configuring and using cloud services. Although I did not have deep practical experience in container orchestration or deployment on specific platforms like Google Kubernetes Engine (GKE), I understood the general concepts of Infrastructure as a Service (IaaS) and Platform as a Service (PaaS).
• Object-Oriented Programming (OOP): Thanks to my university education, I had a broad understanding of the principles of object-oriented programming, which is essential for structuring applications modularly and reusable. This knowledge was crucial for working with NestJS, as this framework promotes a module and controller-based architecture.
• Databases: I had a general understanding of designing, implementing, and managing relational databases. This foundation was crucial for managing the storage and retrieval of user data in the project.

Acquired Knowledge and Skills

During the project's development, I acquired a range of new knowledge and skills that allowed me to perform more complex tasks and handle advanced technologies:

• NestJS Framework: I learned to use NestJS, a progressive Node.js framework that facilitates building scalable and maintainable server-side applications. I understood its module-based architecture, dependency injection, and the use of controllers and services to structure application logic.
• Docker and Docker Compose: I gained experience in containerizing applications using Docker. I learned to create Dockerfiles to build container images and use Docker Compose to define and manage multi-container environments, facilitating consistent development and deployment across different environments.
• Kubernetes and Google Kubernetes Engine (GKE): I acquired practical knowledge about container orchestration with Kubernetes. I learned to deploy and manage applications on Google Kubernetes Engine (GKE), which included configuring clusters, deploying services, and managing resources and scalability.
• Cloud Deployment: I expanded my skills in deploying applications in the cloud, ensuring that the user management microservice was available and functioning optimally in a production environment.
• Continuous Integration and Continuous Deployment (CI/CD): I implemented CI/CD pipelines using GitHub Actions. I learned to automate tests, integrations, and deployments, improving efficiency and reducing human errors in the software development and delivery process.
• Collaboration and Productivity Tools: I used collaboration tools like GitHub and Copilot to improve productivity and code quality. These tools facilitated source code management, code review, and automation of repetitive tasks.

Major Challenges

Developing the user management project with NestJS presented several challenges that required creative solutions and a deep understanding of advanced technologies. Below are the most significant technical and management challenges and the strategies used to overcome them.

Technical Challenges

1. Cloud Deployment:
One of the most important challenges was deploying the application in the cloud, specifically on Google Kubernetes Engine (GKE). Configuring and managing Kubernetes clusters presented a considerable learning curve.

• Cluster Configuration: Configuring Kubernetes clusters efficiently and securely was challenging. It required a detailed understanding of Kubernetes architecture, as well as concepts of nodes, pods, and services.
• Resource Management: Proper resource management, such as allocating CPU and memory to pods, was crucial to ensure optimal performance. This involved learning to use tools like kubectl to monitor and adjust cluster resources.
• Redundancy and Scalability: Configuring redundancy and scalability of the application to handle increased load was a crucial aspect. I used autoscaling configurations and failover policies to ensure high service availability.

2. Container Orchestration:
Using Docker and Docker Compose for application containerization introduced several challenges related to building and managing containers.

• Docker Image Creation: Building efficient and secure Docker images was an iterative process that required constant optimization. This included reducing image size and properly configuring environment variables and volumes.
• Networking and Volumes in Docker Compose: Correctly configuring networks and volumes in Docker Compose to ensure containers could communicate and share data efficiently was a significant technical challenge.

3. Security:
Implementing robust security measures to protect user data and communications was a critical aspect of the project.

• Authentication and Authorization: Configuring secure authentication with JWT and ensuring that only authenticated users could access specific application resources was essential.
• Data Encryption: Implementing encryption of sensitive data, such as passwords, using bcrypt, and ensuring secure communications between services through HTTPS and SSL certificates.

Management and Communication Challenges

1. Project Management:
Although the project was developed individually, efficient time and resource management was crucial to meet project objectives.

• Planning and Organization: Detailed planning of tasks and organizing work in weekly sprints helped keep the project on track.
• Documentation: Maintaining clear and up-to-date documentation of code, configurations, and procedures was essential to ensure project reproducibility and maintenance.

2. Communication:
Although there were no significant challenges in terms of teamwork and communication, clarity in documentation and preparation for future collaborations were important.

• Use of Collaboration Tools: Tools like GitHub and GitHub Actions facilitated source code management and process automation, ensuring that any future collaboration could easily integrate into the workflow.

Overcoming Challenges

1. Utilizing AI Tools:
AI tools, such as ChatGPT and Copilot, were invaluable in overcoming many technical challenges. These tools provided assistance in writing code, problem-solving, and finding optimal solutions to complex issues.

2. Research and Information Seeking:
Actively seeking information in official documentation, development forums, and online communities was essential to overcoming challenges. This proactive approach allowed finding solutions to specific problems and learning from other developers' experiences.

3. Testing and Validation:
Implementing a rigorous approach to testing and validation helped identify and fix errors before they affected production deployment. Unit and integration tests ensured that each system component functioned correctly and integrated seamlessly.

Tools Used

Developing the user management project with NestJS required using various modern tools and technologies. These tools facilitated the implementation of key functionalities, ensured efficient deployment, and provided a collaborative and automated development environment. Below are the main tools and technologies used in the project:

Programming Languages

TypeScript

• Description: TypeScript is a statically typed programming language that builds on JavaScript. It provides additional features like static types and interfaces that help write safer and more maintainable code.
• Project Use: TypeScript was the primary language used to develop the user management microservice. Its ability to catch errors at compile-time and its support for modern JavaScript features facilitated more robust and efficient development.

Frameworks

NestJS

• Description: NestJS is a progressive Node.js framework that facilitates building scalable and maintainable server-side applications. It is based on TypeScript and uses a modular architecture that promotes separation of concerns and code reuse.
• Project Use: NestJS was the chosen framework to structure and develop the microservice. Its modular approach and support for dependency injection allowed for clear code organization and easy integration of new functionalities.

Cloud Services

Google Kubernetes Engine (GKE)

• Description: GKE is a container orchestration service that uses Kubernetes to deploy, manage, and scale containerized applications on Google Cloud.
• Project Use: GKE was used to deploy and orchestrate

the microservice in a production environment. Its ability to manage Kubernetes clusters and scale applications on demand was crucial to ensuring service availability and performance.

Render and Clever Cloud

• Description: Render and Clever Cloud are cloud platforms that offer hosting and deployment services for web applications and microservices.
• Project Use: These platforms were used as additional options for deploying and testing the microservice, providing redundancy and ensuring that the service could be deployed in multiple environments.

Collaboration Tools

GitHub

• Description: GitHub is a collaborative development platform that offers version control using Git, along with tools for code review, project management, and team collaboration.
• Project Use: GitHub was the primary platform for version control and source code management of the project. It facilitated collaboration and code review, ensuring that all changes were documented and managed efficiently.

GitHub Actions

• Description: GitHub Actions is a workflow automation platform that allows integrating and deploying code automatically through CI/CD pipelines.
• Project Use: GitHub Actions were used to implement continuous integration and continuous deployment (CI/CD). This allowed automating tests, integration, and code deployment, improving efficiency and reducing human errors.

Other Tools

Docker

• Description: Docker is a containerization platform that allows creating, deploying, and running applications in containers.
• Project Use: Docker was used to containerize the microservice, ensuring it could run consistently in different environments. Dockerfiles were created to define the container's configuration and dependencies.

Docker Compose

• Description: Docker Compose is a tool that allows defining and managing multi-container applications using a YAML configuration file.
• Project Use: Docker Compose was used to manage the microservice's development environment, allowing easy startup and management of multiple containers, such as the database and application service.

ChatGPT and Copilot

• Description: ChatGPT is a language model developed by OpenAI that provides AI-based assistance, and Copilot is an AI-powered coding assistant tool.
• Project Use: These tools were invaluable in providing assistance in writing code, solving problems, and finding optimal solutions to complex issues. ChatGPT and Copilot helped improve productivity and code quality.

Useful and Challenging Tools

Useful Tools

• ChatGPT: Provided immediate assistance and solutions to technical problems, significantly improving development efficiency.
• Docker: Facilitated application containerization, ensuring it could run consistently in different environments and simplifying the deployment process.

Challenging Tools

• Google Kubernetes Engine (GKE): Despite its power and flexibility, GKE presented a considerable learning curve. Configuring and managing Kubernetes clusters were challenging aspects that required deep understanding and significant effort to ensure successful deployment.

Development Process

The development process of the user management project with NestJS followed an iterative and agile approach, allowing continuous adaptation to requirements and the incorporation of incremental improvements. Below are the main stages and methodologies used and the approach taken for continuous integration and continuous deployment (CI/CD).

Development Methodologies

1. Iterative and Agile Approach

• Description: Agile development focuses on delivering incremental and continuous value with short and frequent iterations. This approach allows quickly responding to changes in requirements and continuously improving the product.
• Implementation: The project was divided into multiple weekly sprints, each with specific objectives and clear deliverables. At the end of each sprint, results were reviewed, and priorities were adjusted for the next sprint.

2. Planning and Organization

• Sprint Planning: At the beginning of each sprint, specific tasks and objectives were defined. This included implementing new functionalities, fixing bugs, and improving infrastructure.
• Kanban Boards: A Kanban board was used to track task progress and ensure sprint objectives were met. This facilitated visibility of the project's status and helped maintain an efficient workflow.

Development and Testing

1. Feature Implementation

• User Registration and Authentication: Endpoints were implemented for user registration and authentication, using JWT for token generation and validation.
• Password Recovery: The password recovery functionality was developed, including sending emails with links to reset passwords.
• Token Validation: Token validation was added to ensure only authenticated users could access protected resources.

2. Unit and Integration Testing

• Jest: Jest was used as the testing framework to write and execute unit and integration tests. Unit tests ensured that each individual component worked correctly, while integration tests verified that different system components interacted properly.
• Test Coverage: Test coverage was monitored to ensure a high proportion of the code was covered by tests, reducing the risk of errors and system failures.

Continuous Integration and Continuous Deployment (CI/CD)

1. GitHub Actions

• Description: GitHub Actions is a workflow automation platform that allows integrating and deploying code automatically through CI/CD pipelines.
• Implementation: CI/CD pipelines were configured in GitHub Actions to automate code testing, integration, and deployment. Every time a commit was made to the repository, GitHub Actions automatically executed the pipeline, ensuring the code passed tests and was deployed without manual intervention.

2. CI/CD Workflow

• Commit and Push: Developers made code changes and pushed them to the GitHub repository.
• Test Execution: GitHub Actions automatically executed unit and integration tests. If all tests passed, the pipeline continued to the next step.
• Container Building: The code was containerized using Docker. New Docker images were built and stored in a container registry.
• Deployment on GKE: The new Docker images were automatically deployed on Google Kubernetes Engine (GKE), ensuring the application was always up-to-date and running optimally.

3. Monitoring and Maintenance

• Prometheus and Grafana: Prometheus and Grafana were implemented to monitor the application's performance and availability. These tools allowed real-time metric collection and historical data visualization, helping identify and resolve issues quickly.
• Alerts and Notifications: Alerts were configured to notify the development team of any critical issues in the system. This ensured a quick and efficient response to any incidents.

Final Results

The development of the user management microservice with NestJS culminated in the creation of a robust and scalable application that meets all the functional and non-functional requirements established at the beginning of the project. Below are the main functionalities implemented, the project's deployment status, the feedback received, and an overall evaluation of the results obtained.

Main Functionalities

The user management project includes several key functionalities essential for any user administration system. These functionalities ensure a smooth and secure user experience, as well as efficient management of user data.

1. New User Registration

• Description: Allows users to register on the application by providing basic information such as name, email, and password.
• Implementation: A registration endpoint was used to validate and store user information in the database. Passwords were encrypted using bcrypt before being stored.

2. User Authentication

• Description: Allows users to authenticate on the application using their email and password.
• Implementation: An authentication endpoint was developed to validate user credentials. If the credentials were correct, a JWT token was generated and returned to the user, which was used to authenticate subsequent requests.

3. JWT Token Generation and Validation

• Description: Ensures that only authenticated users can access protected resources within the application.
• Implementation: JWT tokens were generated during the authentication process and validated on each request to ensure the user had permission to access the requested resources.

4. Password Recovery

• Description: Allows users to recover their password if they forget it.
• Implementation: A password recovery endpoint was created that sends an email to the user with a link to reset their password. The link contains a unique token that allows the user to change their password securely.

5. User Management

• Description: Includes functionalities for updating and deleting user accounts, as well as viewing user information.
• Implementation: Additional endpoints were developed to allow updating user information and deleting accounts, ensuring that only authorized users could perform these actions.

Project Deployment

1. Google Kubernetes Engine (GKE)

• Description: The user management microservice was deployed on Google Kubernetes Engine (GKE), leveraging Kubernetes' orchestration and scalability capabilities.
• Implementation: Kubernetes clusters were configured to manage the application's containers, ensuring high availability and automatic scaling. Docker images were built and deployed using CI/CD pipelines in GitHub Actions.

2. Testing and Production Environments

• Testing Environment: A testing environment was used to conduct thorough testing before production deployment. This allowed identifying and fixing errors before they affected end users.
• Production Environment: The production deployment was done on GKE, ensuring the application was available to users with high availability and performance.

Feedback Received

1. User Feedback

• Description: Users provided feedback on the application's ease of use and functionality.
• Results: The feedback was mostly positive, highlighting the ease of registration and authentication and the quick password recovery process. Some users suggested additional improvements in the user interface and API documentation.

2. Developer Feedback

• Description: Other developers reviewed the project's code and architecture.
• Results: Developers praised the clarity and modularity of the code, as well as the efficiency of the CI/CD process. Some suggestions were made to further optimize performance and improve application security.

Overall Evaluation

1. Meeting Objectives

• Description: The project met all the functional and non-functional objectives established at the beginning.
• Results: All main functionalities were implemented, ensuring the application's security and scalability and achieving a successful cloud deployment.

2. Performance and Scalability

• Description: The application demonstrated good performance and scalability during testing.
• Results: GKE efficiently managed the load and scaled resources as needed, ensuring the application remained available and responsive.

3. Maintainability and Extensibility

• Description: The code is well-documented and structured, facilitating maintenance and future extension.
• Results: The modular architecture of NestJS and the use of TypeScript allowed for clean and easily maintainable development, making it easy to incorporate new functionalities in the future.

Lessons Learned

Developing the user management project with NestJS was an enriching experience that offered numerous opportunities for learning and growth. Throughout the project, valuable lessons were learned that can be applied to future developments and have contributed to a greater understanding of best practices in software development and DevOps. Below are the main lessons learned during the process.

Importance of Planning and Organization

1. Detailed Planning

• Lesson: Detailed planning at the beginning of each sprint and clear definition of objectives and tasks were fundamental to keeping the project on track.
• Future Application: Dedicating enough time to planning and organizing work ensures more efficient execution and reduces the risk of delays and errors.

2. Comprehensive Documentation

• Lesson: Maintaining clear and up-to-date documentation of code, configurations, and procedures was essential to ensuring project reproducibility and maintenance.
• Future Application: Continuing to prioritize detailed documentation not only facilitates maintenance but also allows for more effective collaboration with other developers.

Adopting Agile Methodologies

1. Short and Frequent Iterations

• Lesson: Short and frequent iterations allowed quickly adapting to changes in requirements and continuously improving the product.
• Future Application: Implementing agile methodologies in future projects to ensure continuous delivery of value and greater flexibility in response to changes.

2. Continuous Feedback

• Lesson: Receiving and acting on continuous feedback from users and developers significantly improved the final product's quality.
• Future Application: Fostering a culture of continuous and proactive feedback to quickly identify and resolve problems and continuously improve the product.

Implementing CI/CD

1. Process Automation

• Lesson: Automating processes through CI/CD pipelines in GitHub Actions reduced human errors and improved development and deployment efficiency.
• Future Application: Continue using and improving CI/CD practices to maintain an efficient workflow and ensure code quality in future projects.

2. Automated Testing

• Lesson: Automated tests ensured that new functionalities did not introduce errors and that existing code worked correctly after each change.
• Future Application: Continue prioritizing automated tests and increase test coverage to maintain software stability and quality.

Managing Cloud Infrastructure

1. Orchestration with Kubernetes

• Lesson: Container orchestration with Kubernetes (GKE) proved to be a powerful tool for efficiently managing infrastructure and ensuring service scalability and availability.
• Future Application: Leveraging Kubernetes' capabilities in future projects to manage production environments more efficiently and scalably.

2. Monitoring and Maintenance

• Lesson: Implementing monitoring tools like Prometheus and Grafana allowed a better understanding of the application's performance and quick identification of problems.
• Future Application: Continue using and improving monitoring and alert tools to ensure optimal service availability and performance.

Data Security and Management

1. Data Security

• Lesson: Implementing robust security measures, such as encrypting sensitive data and authentication via JWT tokens, was crucial to protecting user information.
• Future Application: Prioritize security at all development stages and keep security practices and tools up-to-date.

2. Password Management

• Lesson: Implementing secure mechanisms for password recovery improved the user experience and increased system trust.
• Future Application: Continue improving password management and authentication processes to provide a secure and smooth user experience.

Conclusion

Developing the user management microservice using NestJS has been a challenging and enriching project, providing comprehensive experience in implementing modern technologies and best software development practices. Throughout this project, significant objectives were achieved, and valuable lessons were learned that will strengthen future developments.

Project Summary

The project started with a solid foundation in object-oriented programming, cloud services, and databases. From there, significant progress was made in mastering new technologies such as NestJS, Docker, Kubernetes, and GitHub Actions. These tools were essential in building a robust and scalable application that meets the functional and non-functional requirements of efficient and secure user management.

The main functionalities implemented include user registration and authentication, JWT token generation and validation, password recovery, and user information management. Deploying the application on Google Kubernetes Engine (GKE) ensured its availability and performance, while continuous integration and continuous deployment (CI/CD) automated the development process, improving efficiency and reducing errors.

Acknowledgments

This project would not have been possible without the support of various tools and communities. AI tools like ChatGPT and Copilot provided critical assistance during development, facilitating problem-solving and code generation. GitHub and GitHub Actions offered a solid platform for collaboration and workflow automation. I thank all the developers and online communities who shared their knowledge and experiences, helping me overcome technical challenges and continuously improve the project.

I would like to express special thanks to GitHub user María Barros, who guided and provided invaluable support throughout the project. Her guidance and advice were fundamental in overcoming technical challenges and improving the project's quality.

Final Reflection

The user management project with NestJS not only achieved the established objectives but also provided an opportunity to learn and apply advanced technologies in a real context. Detailed planning, adopting agile methodologies, and implementing CI/CD were crucial to the project's success. Additionally, the experience gained in managing cloud infrastructure and data security has laid a solid foundation for future developments.

The positive feedback received from both users and developers confirmed the project's value and provided valuable insights for future improvements. This project has demonstrated the importance of planning, documentation, and using modern tools to create robust and scalable technological solutions.

In summary, the user management project with NestJS has been a formative and rewarding experience that has expanded my skills and knowledge and prepared me to face new challenges in the software development world. I am grateful for this opportunity and excited to apply these lessons in future projects.