DEV Community: Theodo

How to configure CORS for Next.js

Marek Elmayan — Wed, 29 Jan 2025 13:17:18 +0000

Hey 👋,

I've been working on a new package called next-armored that helps you secure your Next.js application. My first release is out, and I'm excited to share it with you. The first tool I added is a middleware to help you configure CORS for your Next.js server.

If you're only interested in how to use it, and how to set it up, you can directly go check the 👉 tutorial section 👈.

Otherwise, let's start with a quick reminder of what Cross-Origin Resource Sharing (CORS) is and when you need to set it up in your Next.js app.

Drop a star on Github ⭐️

Check on NPM 📦

🔒 I - What is the CORS policy ?

Cross-Origin Resource Sharing a web browser security feature

CORS is a security feature that allows you to control which origins are allowed to access your resources. It is enforced by web browsers through Access-control-allow-* headers.

CORS protects users of your app from malicious sites trying to perform operations (like retrieving personal information or account deletion) on your server without their knowledge. While the request will still reach your server, if it is correctly configured, it will attach CORS security headers based on the request’s origin.

If the request comes from your allowed origins, the browser will allow access to the server’s response.
If the request originates from a disallowed or malicious site, the browser will block access to the response.

For example, in the simplest case of a GET request, an attacker could try to retrieve information. For other methods, such as DELETE, the browser sends a preflight request first. Based on the server's CORS configuration, the browser decides whether to proceed with the request.

For more in-depth information, check this article by a friend of mine or the MDN Documentation.
For insights into CORS vulnerabilities, explore Outpost24’s blog.

❓ What is an origin, and how do you differentiate them?

An origin is defined as a combination of the protocol, host, and port of a resource. For example, the origin of https://example.com/index.html is https://example.com. A change in any of these three components results in a different origin.

⚡️ Subdomains change the host, so https://example.com and https://api.example.com are considered different origins.

🛠️ II - When should you configure CORS for your Next.js application?

We saw that https://example.com and https://api.example.com are different origins. So if your server is at https://api.example.com but your frontend app at https://example.com, you need to configure CORS on your server to allow requests from your frontend.

Why is that? Historically, browsers enforce the Same Origin Policy (SOP), allowing requests only between the same origin for security. This policy’s limitations necessitated the introduction of CORS to enable controlled cross-origin requests, such as inter-operability between different subdomains or public APIs.

Now, what about Next.js ? If you use Next.js 13 or later with the app router (e.g., a folder like ./src/app/api), your frontend (e.g., https://your-domain.com) and API (e.g., https://your-domain.com/api) share the same origin. Here, the SOP suffices, and enabling CORS without a valid reason could introduce vulnerabilities.

So, when should you enable CORS?

If your API is accessed by another frontend (e.g., https://admin.your-domain.com).
If your API is public, allowing access from any origin (ensure the API doesn’t handle sensitive data).

👨🏼‍💻 III - How to configure CORS for Next.js with next-armored

📦 Install the next-armored package

Select your favorite package manager (e.g. pnpm) and install the next-armored package by running:

pnpm add next-armored

🥇 Use the CORS middleware as your first middleware

Create or update your ./src/middleware.ts:

import { NextRequest } from "next/server";
import { createCorsMiddleware } from "next-armored/cors";

const corsMiddleware = createCorsMiddleware({
  origins: ["https://example.com", "http://localhost:5173"],
});

export function middleware(request: NextRequest) {
  return corsMiddleware(request);
}

export const config = {
  matcher: ["/api/:path*"],
};

In Next.js, the middleware is executed before every request matching the matcher pattern inside the config object.
For this example, I will suppose that you just created the middleware file. So you just need to match all your api routes.
If you are using the app router, you can match all your api routes by using the '/api/:path*' pattern.
Then import the createCorsMiddleware function from next-armored/cors and pass your configuration object to it. This will allow you to build the cors middleware with the origins you want to allow and pass it to the middleware function.

🎼 Compose your CORS middleware with other middleware

If you already have middleware, compose them as follows:

import { NextRequest, NextResponse } from "next/server";
import { createCorsMiddleware } from "next-armored/cors";

const corsMiddleware = createCorsMiddleware({
  origins: ["https://example.com", "http://localhost:5173"],
});

const otherMiddleware = (request: NextRequest) => {
  console.log("otherMiddleware");
  return NextResponse.next();
};

export function middleware(request: NextRequest) {
  const response = otherMiddleware(request);
  const isApi = request.nextUrl.pathname.startsWith("/api");
  if (isApi) {
    return corsMiddleware(request, response);
  }
  return response;
}

export const config = {
  matcher: ["/api/:path*", "/home/:path*"],
};

If your matcher didn't previously include the api routes, you will have to add it to the matcher.
Then you can call the other middleware first and get the response.
Since the cors middleware should be applied only to the api routes, you have to check if the request is an api request and then apply the cors middleware only in that case. This can be done easily by checking the pathname of the request with request.nextUrl.pathname.startsWith('/api').

Finally, pass the response to the cors middleware as the 2nd argument. In this case, the cors middleware will attach the headers to the existing response instead of returning a new response.

Alternatively, use a utility to chain middleware. Here is a snippet of how you can do it but it shall be adapted to the need of each.

export type CustomMiddleware = (
  request: NextRequest,
  event: NextFetchEvent,
  response: NextResponse
) => NextMiddlewareResult;

type MiddlewareFactory = (next: CustomMiddleware) => CustomMiddleware;

export const chainMiddlewares = (
  functions: MiddlewareFactory[],
  index = 0
): CustomMiddleware => {
  const current = functions[index];

  if (current) {
    const next = chainMiddlewares(functions, index + 1);

    return current(next);
  }

  return (
    _request: NextRequest,
    _event: NextFetchEvent,
    response: NextResponse
  ) => response;
};

export const middleware = chainMiddleware([
  middleware1,
  middleware2,
  corsMiddleware,
]);

⚙️ Full CORS middleware configuration

const corsMiddleware = createCorsMiddleware({
  origins: ["https://example.com", "http://localhost:5173"],
  methods: ["GET", "POST"],
  headers: ["Content-Type"],
  maxAge: 60 * 60 * 24,
  optionsSuccessStatus: 204,
  allowCredentials: true,
  exposedHeaders: ["Content-Type"],
  preflightContinue: false,
});

You have to at least specify the origins option because keeping * by default is not recommended as it can be a security risk. If you are building a public api, you can allow all origins by passing origins: ['*'] but you should be aware of the security implications.

You can define other options as well:

origins - array of origins that are allowed to access the resource. The only one to be required.
methods - array of methods that are allowed to access the resource. Default value to ["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"].
headers - array of headers that are allowed to access the resource. Default value to ["Content-Type", "Authorization"].
maxAge - number of seconds that the results of a preflight request can be cached. Default value to 60 * 60 * 24 (1 day).
optionsSuccessStatus - status code to send for successful OPTIONS requests. Default value to 204.
allowCredentials - boolean value to indicate if credentials are allowed to be sent with the request. Default value to true.
exposedHeaders - array of headers that are exposed to the client. Default value to [].
preflightContinue - boolean value to indicate if it should pass the CORS preflight response to the next handler. Default value to false.

🚫 Enable/disable CORS for specific paths

Enable CORS for specific paths:

const corsMiddleware = createCorsMiddleware(
  { origins: ["https://example.com", "http://localhost:5173"] },
  {
    includes: [{ startsWith: "/api/v2", additionalIncludes: ["example"] }],
  }
);

This will enable CORS for all paths that start with /api/v2 and at the same time include example in the pathname.
api/v2/product/example will have CORS enabled, but api/v1/product/example/test will not.

Disable CORS for specific paths:

const corsMiddleware = createCorsMiddleware(
  { origins: ["https://example.com", "http://localhost:5173"] },
  {
    excludes: [{ startsWith: "/api/restricted" }],
  }
);

In this case, all the routes starting with /api/restricted will not enforce CORS but the SOP (Same-Origin Policy) as the default behavior suggests.

🔮 IV - Why use `next-armored`?

🚀 Easy to use: Create the middleware with few lines.
🔧 Flexible: Customize middleware configurations to fit your needs.
🛡️ Secure: Default configurations are based on best practices and security standards.
✅ Type-safe: Middleware is fully typed and compatible with Next.js and TypeScript. Also it's tree-shakable.
🌐 Open source: Audit, report issues, and contribute.

🔎 V - Next steps for `next-armored`

I started with a cors middleware for Next.js but my goal is to create a bunch of utils to help you secure your Next.js application. So I definitely plan to add more utils and more middleware to this package.
The next one will probably be a middleware to handle Content Security Policy (CSP) headers. But if you have any suggestion, or specific needs, please let me know. 😊

Drop a star on Github ⭐️

Check on NPM 📦

Mastering API Versioning: Strategies for Seamless Frontend-Backend Communication in Mobile Apps

Abdelmoujib MEGZARI — Tue, 09 Jul 2024 09:21:06 +0000

Effective API versioning is essential for maintaining seamless communication between the frontend and backend. This article explains why API versioning is important and analyses various versioning strategies, offering practical insights for backend and mobile app developers. My goal is to equip you with the knowledge to manage API versions efficiently, by ensuring a practical experience for developers while maintaining a seamless user experience. This article is based on our experience with API versioning in the PassCulture app.

1. Context

In the PassCulture app, as with many applications, we control both the frontend and the backend. However, we do not control the version of the frontend that the user is using. This lack of control over the app version necessitates effective API versioning to ensure compatibility and functionality across different user versions. As the app evolves, the API code may change to accommodate new versions of the app, which may not be compatible with older versions.

For example, consider an API endpoint for user authentication that initially requires a username and password.

@app.route('/auth', methods=['POST'])
def auth():
    username = request.json['username']
    password = request.json['password']
    user = User.query.filter_by(username=username).first()
    if user and user.check_password(password):
        return jsonify({'token': generate_token()}), 200
    return jsonify({'error': 'Invalid credentials'}), 401

The frontend (app V1) would send a request to this endpoint with the following code:

fetch("/auth", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    username: "user",
    password: "password",
  }),
});

Now, suppose we want to add two-factor authentication to the app. We could modify the endpoint to require an additional field, otp, for the one-time password.

@app.route('/auth', methods=['POST'])
def auth():
    username = request.json['username']
    password = request.json['password']
    otp = request.json['otp']
    user = User.query.filter_by(username=username).first()
    if user and user.check_password(password) and user.check_otp(otp):
        return jsonify({'token': generate_token()}), 200
    return jsonify({'error': 'Invalid credentials'}), 401

The frontend (app V2) would need to be updated to include the otp field in the request.

fetch("/auth", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    username: "user",
    password: "password",
    otp: "123456",
  }),
});

However, app V1 would not include the otp field, resulting in an error when users on V1 try to authenticate. To avoid this issue, we need to implement API versioning. This way, both versions of the app can interact with the backend appropriately: app V1 can continue to use the original endpoint without otp, while app V2 can use a new version of the endpoint that requires otp. This ensures compatibility and a seamless user experience across different app versions.

2. Alternatives to API Versioning

Is API versioning the only solution to manage changes in the backend code? Not necessarily. Here are some alternatives to API versioning:

a. Forced App Updates

One way to manage changes in the backend code is to force users to update their app. This approach ensures that all users are on the latest version of the app, which is compatible with the latest backend code. However, forced app updates can be disruptive to users and may not always be feasible, especially if users are on older devices or have limited internet connectivity.

Forced updates are a good solution for critical issues, such as security vulnerabilities, that require immediate action.However, they may not be the best approach for non-critical changes or new features.

b. Feature Flags

Feature flags allow you to control the visibility of new features in the app without requiring a new version. By using feature flags, you can gradually roll out new features to users without forcing them to update the app. However, feature flags can be complex to manage and may not be suitable for all types of changes.

If we take the previous example of adding two-factor authentication, we can write the V2 code as follows:

if (featureFlags.twoFactorAuth) {
  fetch("/auth", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      username: "user",
      password: "password",
      otp: "123456",
    }),
  });
} else {
  fetch("/auth", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      username: "user",
      password: "password",
    }),
  });
}

Then we would enable the feature flag when all users have updated their app to V2 and we will simultaniously update the backend code to handle the new field.

We will then need to clean the code to remove the old code and the feature flag.

This approach can be useful for simultaneously rolling out new features across different app versions. However, it requires careful management of feature flags and may introduce complexity to the codebase.

c. API Route Backward Compatibility

Another approach is to maintain backward compatibility in the API routes. This means that the backend code is designed to handle requests from older app versions, even if the route has been updated. For example, the backend code can check if the request contains the otp field and handle it accordingly.
Example:

  @app.route('/auth', methods=['POST'])
  def auth():
      username = request.json['username']
      password = request.json['password']
      otp = request.json.get('otp')  # Use get to avoid KeyError
      user = User.query.filter_by(username=username).first()
      if user and user.check_password(password) and (otp is None or user.check_otp(otp)):
        token = generate_token()
        if opt is None:
          limit_access_to_some_features(token)
        return jsonify({'token': token}),200

      return jsonify({'error': 'Invalid credentials'}), 401

This approach can be useful for minor changes. However, it can lead to complex code and potential bugs if not managed properly. It also requires careful testing to ensure that backward compatibility is maintained. It also requires the developer to clean the code once the old version of the app is no longer used.

3. Possible Versioning Solutions

When managing multiple API versions, there are several strategies to consider. Mainly three strategies come to mind:

a. Code level versioning

This solution consists of managing the different versions of the routes in the code.
In this case, the different route versions are just different routes in the code. We declare new routes with the version included in the URL.

Example:

@app.route('/v1/auth', methods=['POST'])
@route.deprecated
def auth():
    pass

@app.route('/v2/auth', methods=['POST'])
def auth_v2():
    pass

In this case, The app specifies the version of the route it wants to use in the URL.

For example the app V1 will use the route /v1/auth as follows:

fetch("/v1/auth", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    username: "user",
    password: "password",
  }),
});

The app V2 will use the route /v2/auth as follows:

fetch("/v2/auth", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    username: "user",
    password: "password",
    otp: "123456",
  }),
});

b. Infrastructure level versioning

This solution consists of deploying multiple API instances in parallel on different pods. Each instance corresponds to a version of the API. The app specifies the version of the API it wants to use in the request. The infrastructure is responsible for routing the request to the correct API instance.

A Pod is a group of one or more containers, with shared storage/network resources, and a specification for how to run the containers. Pods are the smallest deployable units of computing that can be created and managed in Kubernetes. Read more about Kubernetes Pods.

In this case, whenever the developer wants to create a new version of a route, he just needs to change the code without thinking about the old version. The old version of the route will still be available as long as the old pod is still running, and the new version will be available as soon as a new pod is deployed with the latest API code.

When a pod is no longer used, it can be deleted. This will delete the old version of the route. Deleting the pod will depend on the oldest version of the app users can still use.

In general, a git branch is created for each version of the API. In case of critical issues, modifying the old code can be done by cherry-picking the fix commit to the versions branch and redeploying it from that branch.

In contrast, to code level versioning, infrastructure level versioning doesn't require the developer to clean the code when the old version is no longer used. Additionaly, infrastructure level versioning ensures that a change in the code will not affect the old versions of the app, thus requiring no quality check on the old versions of the app.

c. Backend For Frontend (BFF) level versioning

This type of versioning is a compromise between the two previous solutions. It consists of splitting the backend into two parts: the main backend and the BFF. The main backend is responsible for the business logic and the data access. The BFF is responsible for calling the main backend and formatting the data in the way that each app version expects. The BFF is versioned, and the app specifies the version of the BFF it wants to use in the request.

The main contrast between this solution and the infrastructure level versioning is that it ensures that a change in the business logic will affect all versions of the app. While keeping the advantage of the infrastructure level versioning of not needing to clean the code.
However it requires a good separation of the business logic from the data formatting and the data access.

4. Key Considerations

In order to choose the best solution for API versioning, we consider the following key criteria:

a. Correcting Critical Bugs Across Versions

It’s crucial to be able to check which versions are affected by a bug or a vunerability. It’s also important to be able to fix the bug in all affected versions.

b. Same business logic for all versions

It’s important to ensure that a change in the business logic will affect all versions of the app. Since this can create inconsistances between the different versions of the app. And a use of an old business logic can be a critical security issue.

c. Maniability

It should be easy for the developers to manage the multiple versions:

Easy to create, update or delete a version of a route
Easy to mark versions as deprecated
Easy to update the app's code to use a new version of a route

d. No Version inter-dependency

When managing multiple versions, it's important to be able to make changes to a specific version without affecting others. For example, if there is a bug in version 1.0.0, you should be able to fix it without impacting version 2.0.0. Releasing a new version should not affect existing versions, thereby avoiding the risk of introducing new bugs into old versions. This approach ensures that older versions remain stable and reliable, eliminating the need for unnecessary quality checks on all versions with every release. This strategy should allow for better control over the quality of older versions.

e. Traceability

It should be easy to know which version of a route is being used by which version of the app and vice versa.

This allows for better tracking of the usage of the different versions of the routes, helping on decisions for when to delete or update existing routes.

f. Deployment process

The chosen solution should make the deployment process simple. It should be easy to deploy a new version of a route.

g. Set Up

The effort required to set up the solution should be taken into account. The solution should be easy to set up and maintain.

5. Evaluation of the different solutions

Criteria	Infrastructure Level Versioning	Code Level Versioning	BFF Level Versioning
Correcting Critical Bugs	▲ Capable of addressing fixes; requires identifying and applying fixes individually to each version, followed by re-deployment for each.	✔ Single fix in the codebase with a single re-deployment.	▲ A single fix is possible if it pertains to business logic; otherwise it's similar to infrastructure level versioning.
No Version inter-dependency	✔ Completely isolated instances	✘ Changes in the code might impact multiple versions.	▲ Changes in the main backend (business logic) can influence multiple versions, but a change to a route has no impact on other versions.
Maniability: route creation	✔ Simple route creation.	✔ Simple route creation.	✔ Simple route creation.
Maniability: route update	▲ Update requires some effort: need to modify the concerned route on the different branches associated to each version that we want to update. requires a redployment of the pods associated to the concerned versions.	✔ Simple route update	▲ Update requires some effort when the change is on the BFF level. Easy to update when the change is in the main backend.
Maniability: route deletion	✔ Deletion doesn't require developer intervention: The pod is automatically terminated when no routes for the specific version are in use. However, you cannot delete a single route without deleting all routes for that version(The whole pod is terminated).	▲ Requires developper intervention. Can delete a single route of a specific version.	✔ Doesn't require developper intervention, Can not delete a single route without deleting all routes for that version.
Traceability	✔ Easy to track.	✔ Easy to track.	✔ Easy to track.
Deployment Process	✘ Complex: multiple instances manage	✔ Simple: single deployment process	✘ Complex: multiple instances manage
Set Up	✘ Complex	✔ Simple	✘ The most complex: requires a good separation of the buisness logic, and the features

✔: Solution meets the criterion effectively.
▲: Solution moderately satisfies the criterion.
✘: Solution does not meet the criterion or makes it more complex.

6. Our choice

After evaluating the different versioning strategies, we opted for code-level versioning. Here’s why:

Complexity: We ruled out infrastructure-level versioning due to its complexity in setup and maintenance. Additionally, it was crucial for us that any change in business logic affects all app versions, which infrastructure-level versioning does not guarantee.

Independence: While BFF level versioning provides some degree of version independence, it also introduces significant setup and maintenance challenges. Furthermore, it doesn’t ensure full version independence.

Business Logic Separation: By effectively separating the business logic from data formatting and access, code-level versioning can offer the same benefits as BFF level versioning. In our scenario, complete independence between versions is not critical, thanks to our comprehensive test suite that catches most bugs introduced by changes. Additionally, we are working towards automating end-to-end tests to identify critical bugs in older app versions.

Practicality: Given our context, simplicity is key. Code-level versioning emerged as the most straightforward and practical solution, aligning well with our testing strategies and maintenance capabilities.

Therefore, we settled on code-level versioning as it meets our needs without adding unnecessary complexity.

Conclusion

I have explained the importance of API versioning and presented three solutions—infrastructure-level, BFF-level, and code-level versioning—each with its own pros and cons.

If you are managing an application where backend and frontend evolve independently, versioning is crucial. I suggest starting by determining your priorities and assessing the most important criteria, and I hope this article will help you to select the best versioning strategy for your needs.

Learn how to implement VAT for your e-commerce website with Sylius in 6 minutes

Charlier — Wed, 24 Apr 2024 14:59:32 +0000

If you are using Sylius for your e-commerce website, then you will inevitably come across the subject of Value Added Tax (VAT). As a software engineer at Theodo, I have spent countless hours customizing every one of its aspects, for it to behave in the exact way I wanted it to.

So you don't need to! Here are the main steps you can follow to avoid wasting any time!

Table of contents:

Setting up your first VAT engine 🔧
1. Tax category 🏷
2. Zone 🌍
3. Tax rate 💸
How does Sylius VAT engine works? 🧠
1. The Order Processor ✨
2. The Applicators 👨‍💻
3. The Calculators 🧮
My tips and tricks 🥇
1. Changing the default address 🏠
2. Multiple zones for one country 🌐
Conclusion: customize your website 🔥

Setting up your first VAT engine 🔧

Sylius VAT handling revolves around three distinct entities, allowing it to choose which rate it has to apply. One of these entities is linked to the article - the tax category, another is linked to the customer - the zone, and the last one connects the latter two - the tax rate.

Tax Category 🏷

The tax category is just a category for your product, which defines which rates should be applied. It allows you to sell products which aren’t taxed for the same rate - for example, clothes and food.
You’ll need to add all of your tax categories to your fixtures (the base data you want to have every time you restart your shop - see Sylius documentation):

    fixtures:
        tax_category:
            options:
                custom:
                    food:
                        name: 'Food'
                        code: 'food'
                    clothes:
                        name: 'Clothes'
                        code: 'clothes'

Zone 🌍

The zone is a Sylius concept which is way broader than the VAT: it is either a group of countries, a group of provinces, or a group of other zones. Here, what we will be looking at is the zone of the customer. A zone has an interesting characteristic, called scope, which defines what it can be used for: ‘shipping’, ‘tax’, or ‘all’. A zone with a shipping scope is only used to calculate shipping methods and fees, whereas a zone with tax scope is only used to calculate VAT. If the scope is all, the zone will be used for both.
You’ll need to define the different zones that you need to your fixtures:

    fixtures:
        geographical:
            options:
                countries:
                    - 'FR'
                    - 'DE'
                    - 'US'
                    - 'GB'
                zones:
                    EUROPE:
                        name: 'Europe'
                        scope: 'tax'
                        countries:
                            - 'FR'
                            - 'DE'
                    AMERICA:
                        name: 'America'
                        scope: 'tax'
                        countries:
                            - 'US'

Tax Rate 💸

Finally, a tax rate is an entity that defines all the necessary information for Sylius to calculate the amount of taxes your customer should pay. This must at least contain:

a tax category: which of the products are concerned
a zone: which of your customers are concerned
an amount: how much is the VAT ? For 20%, you have to write 0.2.
a calculator: how should the VAT be calculated ? Sylius provides a calculator named default that does a basic division, but you can define and provide others if you need to.
a isIncludedInPrice boolean: is the VAT included in the price that you defined initially for the article ?

The last field, isIncludedInPrice, might seem enigmatic at first glance, however it is quite simple. It defines how the VAT should be calculated. For example, for a T-shirt costing 12€, and a tax rate of 20%:

if isIncludedInPrice is false, that means that the 12€ was tax free, and the final price is 14.40 € (2.40€ of VAT included)
is isIncludedInPrice is true, that means that the 12€ was tax included, and Sylius will keep track of 2€ of this being taxes, without changing the total amount.

You’ll need to define a tax rate for every couple of zone and tax category:

    clothes-europe:
        code: 'clothes-europe'
        name: 'Tax rate for clothes in Europe'
        category: 'clothes'
        zone: 'EUROPE'
        amount: 0.2
        calculator: 'default'
        included_in_price: true
    clothes-america:
        code: 'clothes-america'
        name: 'Tax rate for clothes in America'
        category: 'clothes'
        zone: 'AMERICA'
        amount: 0.15
        calculator: 'default'
        included_in_price: true
    food-europe:
        code: 'food-europe'
        name: 'Tax rate for food in Europe'
        category: 'food'
        zone: 'EUROPE'
        amount: 0.2
        calculator: 'default'
        included_in_price: true

If you only need to setup a basic VAT engine, without any added complexity, then you’re good to go! Jump to your code editor and try it out! Sylius will handle all the calculations on its own, and add the correct adjustments (modifications of price) to your articles automatically.

How does Sylius VAT engine works? 🧠

However, if you need a custom behavior of your engine (for example: billing a commission and adding VAT to it), or if you want a deeper understanding of this system, I’ll dive further into the engine now.

The Order Processor ✨

The VAT engine is included in the OrderProcessor: a list of operations that are executed every time a cart is modified (for example, it is triggered by adding an item to cart, changing the shipping address, etc…). That means that the VAT will be erased and recalculated every time your customer modifies its cart. This includes recalculating the shipping fees, the VAT, ensuring that every item is sold at the correct price, etc...

When the OrderProcessor asks for a VAT calculation, it calls a service called OrderTaxesProcessor, whose only goal is to call some Applicators. Applicators are where the magic happens, and are responsible for finding the correct tax amount, and adding this amount to the order. Each object that can generate VAT has a dedicated Applicator: there is one for the OrderItem, one for the shipping fees, etc…

Note that, prior to asking for a VAT calculation, the OrderProcessor calls another service which removes the taxes from the order. That allows us to calculate the VAT again, from an empty and controlled state.

Schema of the different steps of the Order Processor from Sylius documentation

The Applicators 👨‍💻

An applicator will do the following steps:

Find the zone which corresponds to the current cart. If the order has a billing address, it uses it to find the zone. Otherwise, it uses the default zone of your channel.
Find the tax category of every item in the cart.
For each item, look for a tax rate that has both the correct tax category and zone.
If a tax rate is found, adding VAT adjustments with the correct rate, otherwise do nothing.

The Calculators 🧮

The Sylius VAT calculators are some pieces of code that handle the calculation of the correct tax amounts. They are given a price and a tax rate, and they return the correct amount of vat. Sylius, by default, provides one that calculates the VAT the way you expect it (a basic division), but if you need to implement some custom logic here (for example, no VAT at all for product that cost less than 5€), you can do so easily.

The only things to do are:

Create a service which implements the Sylius CalculatorInterface with your own logic
Register it in your services with the dedicated tag and the name you want to use in your fixtures.

    // NoTaxesIfPriceLowerThanFiveCalculator.php
    class NoTaxesIfPriceLowerThanFiveCalculator implements CalculatorInterface
    {
        public function calculate(float $base, TaxRateInterface $rate): float
        {
            if ($base <= 5) {
                return 0;
            }

            if ($rate->isIncludedInPrice()) {
                return round($base - $base / (1 + $rate->getAmount()));
            }

            return round($base * $rate->getAmount());
        }
    }

    // services.yaml
    App\Taxation\Calculator\NoTaxesIfPriceLowerThanFiveCalculator:
        tags:
            - { name: 'sylius.tax_calculator', calculator: 'no_taxes_if_price_lower_than_five' }

My tips and tricks 🥇

Now you know almost everything that you need to customize and code tax calculations with Sylius with ease. However, there is still some details and tricks that I discovered which can be useful to anyone working with this system.

Changing the default address 🏠

By default, to calculate the zone which corresponds to the current cart, Sylius uses the billing address. However, if you want it to use the shipping address instead, you don't have to add any code or to overwrite anything. There is an optional parameter that you can add to your yaml configuration, and which does exactly that:

    sylius_core:
        shipping_address_based_taxation: true

Multiple zones for one country 🌐

If a country is attached to multiple zones having 'tax' as their scope, then issues may arise. Indeed, Sylius expects a country to have a single zone available for tax, and, if it finds more that one, it will randomly return one of them. This will leave you with unexpected behaviors, which can cause severe bugs. Instead, you should split your countries into multiple zones, until no country is present multiple times.

Example: I deliver food and clothes in FR, IE (Ireland) and UK. Food has a 10% rate in every country, but clothes are only taxed in FR with 15% rate. I would like to create a zone containing every country (EUROPE), and a zone containing only FR (FRANCE), so that I can create two tax rates:

one for EUROPE, food, with a rate of 10%
the other for FRANCE, clothes, and 15%.

However, this won't work, as France is included in two different zones. Instead, I have to declare a zone containing IE and UK (GREAT-BRITAIN) and a zone containing FR (FRANCE), and then create 3 tax rates:

one for GREAT-BRITAIN, food, with a rate of 10%
one for FRANCE, food, with 10%
and one for FRANCE, clothes, and 15%

Conclusion: customize your website 🔥

You can now easily twist the Sylius VAT engine to your needs, and explore its features in detail. And the best part is that you can replace any part of this engine! Thanks to the Sylius architecture using numerous specialized services, you can easily override and replace any code that doesn't exactly meet your needs, allowing you to create the e-commerce website that exactly fits your business.

How to generate Typescript interfaces from your Spring Boot backend

jeremiec — Tue, 09 Apr 2024 12:02:00 +0000

Starting a full stack project with Spring Boot and a modern frontend framework like React in Typescript, you rapidly fall into the issue of defining your interfaces twice:

Once on Spring Boot side where you create your response/request DTO for your controllers,
and again on the frontend where you have to write those interfaces again in Typescript this time.

Not only is this poor developer experience, leading to a loss in productivity, it simply leads to bugs as you can easily forget to upload your interfaces on one side.

Great news! Avoiding this pitfall is actually possible and easy ?! with a quick setup, which is the topic of the following guide, let's go!

How can we generate Typescript interfaces from our Spring Boot backend

Advancement in code generation and support of standards like Swagger/Open API to describe REST APIs allow for effective code generation. Meaning we only need to set up a proper Swagger on our backend api in Spring Boot to benefit from interface generation for all our interfaces.

In practice

We will need to expose a Swagger on Spring Boot application and then to configure our code generation tool to target it.

Setting up SpringDoc

To generate our Swagger we will use SpringDoc as SpringFox is outdated and less maintained. (You can even relatively easily migrate from SpringFox to SpringDoc)

Setting up SpringDoc with Maven is classic, get the latest version, add it to your pom:

<!-- Swagger -->
    <dependency>
      <groupId>org.springdoc</groupId>
      <artifactId>springdoc-openapi-ui</artifactId>
      <version>1.6.9</version>
    </dependency>

If you are using some sort of authentication, you will need to allow unauthenticated access to the Swagger, eg:

// ... rest of your auth
 @Override
  public void configure(WebSecurity web) throws Exception {
    // Allow Swagger to be accessed without authentication
    web
      .ignoring()
      .antMatchers("/api-docs")
      .antMatchers("/api-docs/swagger-config")
      .antMatchers("/swagger-ui.html")
      .antMatchers("/swagger-ui/**");

Which you can limit to dev environment if you are not comfortable allowing your backend users to discover your API endpoints using Swagger, to do so use profiles/environment variables as you can easily disable SpringDoc.

Now start or restart your Spring Boot application, you should be able to get your Swagger (JSON) on http://server:port/context-path/v3/api-docs, for the application created for this guide it's: http://localhost:8080/guide/v3/api-docs.

One additional step is to configure your Swagger to regroup enums usage definition (meaning when you use an enum in two DTO you will expose only one enum definition in Swagger instead of duplicating the definition):

@Configuration
public class OpenApiConfig {
  static {
    // use Reusable Enums for Swagger generation:
    // see https://springdoc.org/#how-can-i-apply-enumasref-true-to-all-enums
    io.swagger.v3.core.jackson.ModelResolver.enumsAsRef = true;
  }

  // ... you can also describe your api bellow
  @Bean
  public OpenAPI openAPI() {
    return new OpenAPI()
      .info(
        new Info()
          .title("My shiny API")
          .description("My shiny API description")
          .version("v0.0.1")
      )
      .externalDocs(
        new ExternalDocumentation()
          .description("Documentation")
          .url("https://my.shiny.api.doc/")
          // can simply be a link to a README
      );
  }

This setup is enough for code generation, however I recommend to dive deeper into SpringDoc, it also offers you an UI http://server:port/context-path/swagger-ui.html which can be an excellent way to document your api for your clients.

Setting up Orval

Orval is a recent code generator that propose code generation up to API client generation with the tool of your choice: fetch, Axios, React Query, SWR. For this guide, I will cover interface generation only, however if you are starting a project, I highly recommend setting up client generation as well.

First install Orval as a dev dependency on your project:

npm install orval --save-dev

then create an orval.config.ts file in your frontend:

import { defineConfig } from "orval";

export default defineConfig({
  evo: {
    output: {
      mode: "tags",
      schemas: "model/api",
      mock: false, // enable/disable test mock generation
      // I recommend enabling this option if you generate an api client
      prettier: true, // recommended if you use prettier
      clean: true, // recreate the whole folder (avoid outdated files)
    },
    input: {
      // use your own Swagger url: http://server:port/context-path/v3/api-docs
      target: "http://localhost:8080/guide/v3/api-docs",
    },
  },
});

Once your backend is running, you can generate your interface running npx orval --config ./orval.config.ts.

I recommend setting up a script in package.json:

{
  ...
  "scripts": {
    ...
    "gen:types": "orval --config ./orval.config.ts"
  }
}

That's it! Now you only need to launch this command to update your frontend interface when you modify your backend, no more duplication 🎉

Going further

You can go a bit further as mentioned in passing in this guide:

SpringDoc provides a great UI to self document your API, and you can customize the Swagger generated for your clients,
Orval has many features including client generation, creating mock for your API, I highly recommend looking into it,
the next step would be to automate client side type generation in CI to test that every pull request has up to date types.

Mastering File Upload Security: DoS and Antivirus

Marek Elmayan — Wed, 20 Mar 2024 13:45:00 +0000

TL;DR:

Mitigate Denial of Service (DoS) threats by imposing restrictions on both the size of files and the quantity of uploads permitted on your server.
Implement antivirus scanning for files that users download, ensuring protection against malware infections.

File Upload vulnerabilities and security challenges

Incorporating a file upload feature into your website introduces a spectrum of vulnerabilities that can compromise not only your servers but also the integrity and safety of your users' data.

This article aims to shed light on the critical security challenges associated with file uploads and offers a roadmap to safeguard your digital infrastructure against malicious attacks. We'll explore strategies to prevent attackers from exploiting file upload functionalities to overload and destabilize your servers. Additionally, considering the risks posed when users download files uploaded by others, we emphasize the importance of integrating robust antivirus scanning processes.

⚡️ Denial of Service (DoS) attacks

A Denial of Service (DoS) attack is a malicious attempt to disrupt the normal functioning of a network, service, website, or computer system by overwhelming it with a flood of traffic or requests. The primary objective of a DoS attack is to make a target system or network unavailable to its users, thereby denying access to legitimate users. In our case, uploading specific files can result in service disruptions for unsafe server configurations.

Considering a file upload stream what are the possibilities to block the traffic?

Voluminous File Upload: This vulnerability occurs when an attacker uploads an extremely large file, often much larger than what the system is designed to handle. Such a file can consume all available server resources, such as disk space, memory, and processing power, causing the system to slow down or even crash.

Multiple Small File Upload: In this scenario, an attacker uploads a large number of relatively small files. While individual files may not be large, the cumulative impact can lead to resource depletion or storage space exhaustion.

🛡️ How to prevent file upload DoS attacks from happening?

To deal with these vulnerabilities, it's essential to implement proper security measures in the backend part of your file upload feature. It’s relatively easy and straightforward to do.

Set appropriate file size limits to prevent the upload of excessively large files that could overload your system.
Set a limit to the number of simultaneous files that can be uploaded at the same time.

Here is an elementary Typescript implementation example:

const maxFiles = 6;
const maxFileSize = 50 * 1024 * 1024; // 50 MB limit

const uploadedFiles: File[] = req.body.files;
// Assuming 'files' is an array of File objects in the request body

// Check file size and number of files
if (uploadedFiles.length > maxFiles) {
  return res.status(400).json({ error: "Too many files uploaded." });
}

for (const file of uploadedFiles) {
  if (file.size > maxFileSize) {
    return res.status(400).json({ error: "File size exceeds the limit." });
  }
  // Handle the file, e.g., save it to the server
}

return res.status(200).json({ message: "Files uploaded successfully!" });

More general tips to prevent any type of DoS attacks are:

Implementing rate limiting for file uploads to prevent the rapid uploading of numerous small files. This helps control the flow of incoming data.
IP blocking → blocking traffic from known or suspected malicious sources to prevent DoS traffic from reaching its target. You can find lists of abusive IPs, for instance on abuseipdb.com
Using a Content Delivery Networks (CDNs) such as Cloudflare, can help absorb large amounts of traffic and mitigate the impact of DoS attacks.

Additional benefits of DoS protection measure: 💰 & 🌱

Using smaller and fewer files can bring also other benefits to you and the environment.

With less quantity to store on your server, you will have better performances and less to pay at the end of the month.
Moreover, less storage usage will reduce the environmental impact of manufacturing, operating, and cooling data centers.
Sending voluminous files is also costly so even though front-end file size validation isn’t enough for security measure it can help to reduce the environmental impact of the uses.

💣 What is a Zip Bomb? A type of DoS attack

A Zip Bomb, also known as a Zip of Death or decompression bomb, is a malicious archive file designed to crash or render useless the program or system reading it. It's a type of DoS attack. The concept behind a Zip Bomb is to exploit the file compression algorithm's ability to highly compress a large amount of data. A Zip Bomb file appears small and innocuous but contains an enormous amount of data when decompressed.

When a targeted system tries to decompress such a file, it can exhaust system resources due to the disproportionally large size of the decompressed data compared to the compressed file. For example, a Zip Bomb could be a file that is only a few kilobytes in size but expands to several gigabytes or even terabytes when decompressed. This can lead to system crashes, slowdowns, or can render the system inoperable, thus achieving the attacker's goal of denying service to legitimate users.

Zip Bombs are not only limited to Zip files; they can be created in any file format that supports compression, such as RAR or 7z. Accepting an archive file is always a delicate matter, and decompressing it on your server even more so. The defense against Zip Bombs involves careful scanning and validation of input files, limiting the maximum size of decompressed data, and using updated antivirus and anti-malware programs capable of detecting such threats.

🕵️‍♂️ Antivirus scanning for user protection

The purpose of uploading files is very often to share them with other people thus another user is supposed to download the file afterwards. These users can inadvertently become targets of malicious activity when using file upload features on web applications. In many cases, attackers may attempt to exploit the trust of users by uploading files that appear harmless but, in reality, contain malware or other malicious content. This can include infected documents, executables, or scripts. When these files are downloaded and executed on a user's system, it can lead to compromised security and potential data breaches.

If the uploaded files are retrievable, to safeguard both your platform and mainly your users, it is crucial to run an up-to-date virus scanner. Virus scanners are very adept at spotting malicious files masquerading as a different file type, invisible to most unaware users. This proactive measure can automatically check each uploaded file for potential threats, helping ensure that users are not unwittingly exposed to malicious content.

Running an antivirus can also help you secure your server, because it can detect potentially malicious files that succeeded in bypassing all your previous security checks.

Before really uploading a file you can isolate it and run an antivirus to identify potential threats, then only finally uploading it. This way all potential threats are being neutralised before having access to sensitive data.

⭐️ Recommendations to go beyond simple antivirus scanning

For maximal safety, you can implement multi-scanning. For each file run multiple anti-malware engines, which can be parallelised, in order to get the highest detection rate and the shortest window of exposure to malware outbreaks. Here are some reasons:

Malware can bypass a single antivirus engine.
Different anti-virus vendors have different response times to outbreaks due to their location and focused markets.
False positives in virus detection are a common side-effect of any malware scanning solution.

Lastly, if you have to choose between an Online and an Offline antivirus, you should probably prefer an online solution.

If you use offline antivirus software, you have to download and install it in your system. While online virus scanner does not require any storage space.
Real-time Protection & Regular Updates: Online antivirus software constantly updates its virus definitions and scans your system in real-time, providing immediate protection against the latest threats. Online antivirus solutions automatically receive updates and patches to keep up with the evolving threat landscape.
Cloud-Based: Online antivirus relies on cloud servers to analyse files and detect malware. This reduces the strain on your computer's resources and can be more efficient since the processing part is usually done on high-speed cloud-based servers.
Enhanced Detection: Online antivirus solutions often use behavioural analysis and machine learning algorithms to identify previously unseen malware, improving their ability to detect new threats.
Less Control & Privacy: Data sent to external servers and no customisation according to personal preferences are the main drawbacks that would make you prefer an offline solution.

My recommendation would be to use ClamAV an open source antivirus engine. It is a versatile tool designed to detect multiple types of threats from numerous file formats and other use cases (cross-platform, integration such as mail server). Finally, it is updated on a daily basis, ensuring protection against the latest known threats. This rapid update cycle is crucial for an antivirus tool to be effective.

Setting up AWS antivirus scanning for files in S3 Buckets

You can find a guide on the AWS blog, detailing how to integrate Amazon S3 Malware Scanning into your web App. Amazon Simple Storage Service (Amazon S3) is a highly scalable object storage service that allows file storage. Currently, when a file is uploaded to an S3 Bucket, two scanning engines are available: Sophos and ClamAV, to detect malicious ones. Additionally, both engines can be used together for an even higher safety.

Asynchronous antivirus scanning to enhance performance and UX

Running an extensive antivirus engine can be quite time-consuming thus implementing asynchronous antivirus scanning can significantly enhance both performance and user experience in systems that require real-time or near-real-time processing. This approach involves decoupling the scanning process from the main application flow, allowing the application to continue executing without waiting for the scan to complete.

When a user uploads a file, the server immediately acknowledges receipt and begins the upload process, providing a responsive experience.
The file is being processed in the background, scanned and uploaded if successful or either deleted or quarantined if not.
Send a status update on the upload process to the user, such as through email notifications, dashboard updates, or real-time alerts using web sockets.

🏎️ Race conditions applied to file upload

Race conditions in cybersecurity refer to a scenario where the outcome of a process is unexpectedly altered due to the timing of events not happening as planned. This occurs in a concurrent environment where multiple processes or threads execute in overlapping timeframes, leading to conflicts in accessing or modifying shared resources. The security risk arises when an attacker can manipulate the timing of actions to gain unauthorized access or privileges, modify data, or exploit system functionality in unintended ways.

In the context of file upload vulnerabilities, race conditions can be particularly concerning. There are multiple scenarios where the file security checks could be bypassed. Here is one example:

Imagine a web application that allows users to upload files. The application checks the file for security purposes (e.g., ensuring it's not a malicious executable) before moving it to a permanent storage location. A race condition vulnerability could occur if an attacker manages to manipulate the timing of the upload process.

Initial Upload: The attacker begins by uploading a benign file that passes the application's security checks.
Timing Manipulation: Right after the initial check but before the file is moved to its permanent location, the attacker quickly replaces the benign file with a malicious one, exploiting the time gap in the process.
Execution of Malicious File: The application, believing the file to be safe, moves it to permanent storage, from where the attacker can trigger the execution or processing of the malicious file.

This exploitation leverages the race condition to bypass security checks, allowing the upload and execution of potentially harmful files.

You can try yourself exploiting a file upload race condition vulnerability, of a different kind, on this Portswigger’s Lab.

Conclusion:

I hope this article has provided you with a comprehensive understanding of the security challenges associated with file uploads and the strategies to mitigate these risks. By implementing the recommended security measures, you can ensure that your file upload feature is robust, resilient, and secure, protecting your servers and users from potential threats.

If you'd like to find out more about file type validation, take a look at the first article in this series: Mastering File Upload Security: Understanding File Types.

API Load Testing: Enhance Your Skills with Locust

Paul Royer — Wed, 28 Feb 2024 13:30:00 +0000

Load testing is a serious job and doing it poorly can have disastrous consequences for big companies' production servers. But load testing is not only for experts in massive companies! You may want to quickly check that your server performances are appropriate, using only your own computer.

Locust is a perfect tool to use on such occasion:

A great user interface, allowing you to launch tests and monitor results in real-time, directly from your web browser.
Writting scripts in Python, perhaps the most popular language according to, for example, the top programming language IEEE study.
Installable in seconds, free and open source

Start load testing with Locust!

First, you need an API to load test. Let's say you have an amusement park application. It's running on http://localhost:8080 and exposing the route GET /bookings with two query parameters: age and date.

To begin with Locust, you basically need a dozen seconds. With pip already installed, just run:

pip3 install locust

Then, you need to write a locust file, describing how users will interact with your API. Let's create a basic one, under the name locustfile.py:

from locust import HttpUser, between, task

class User(HttpUser):

    wait_time = between(1, 2)

    @task
    def get_booking(self):
        age = 19
        date = "2024-06-08"
        self.client.get(f"/bookings?age={age}&date={date}")

The User class represents the expected behaviour of a user. For each user created, Locust will execute on a loop all the tasks defined. In this simple case, we have only one task and the requested URL will be always the same as age and date variables are hardcoded. We just set a pause of 1 to 2 seconds between each user loop.

Set-up is done, let's swarm into our API using Locust! In your terminal, in the folder where your locustfile is located, simply run:

locust

On your browser, head to http://0.0.0.0:8089/ and fill up the three parameters before launching the test:

Press "Start swarming" and tadaa! ✨ We can now observe the response time of our API fluctuate in real-time:

As the number of users grows:

Once the test is done, you can download a report in HTML format. Allowing you to quickly share the results visually with your team.

Benefiting from Locust plugins to deliver more value

Here, we load-tested our API in a basic way (one endpoint, always the same request parameters). We can do a lot more, and most of the time we must!

Let's say your API is enjoying some success. To handle the increasing load, you implement some cache on your endpoint, allowing fast response to common requests. It's nice, but if you keep having only one request when load testing, cache will handle all your requests. So your server will be able to accept way more requests than in a real situation, where all requests could be different and cache would miss some of them.

To solve this issue, we should vary the parameters of the requests we send, that is age and date. Here is one of the many possibilities, using the CSV reader plugin for Locust:

First, we need to create a CSV file listing all the test cases we want to use, for example in a file named booking-param.csv:

12, 2024-06-08
20, 2024-06-08
12, 2024-06-09
20, 2024-06-09

You can use a little script to build this CSV by making the cartesian product of all your parameters' possible values.

Then, we modify the locust file to import our parameters from the CSV file:

from locust import HttpUser, between, task
from locust_plugins.csvreader import CSVReader # add import to the CSV reader plugin

booking_param_reader = CSVReader("booking-param.csv") # read the CSV you created

class User(HttpUser):

    wait_time = between(1, 2)

    @task
    def get_booking(self):
        booking_params = next(booking_param_reader)
        age = booking_params[0]
        date = booking_params[1]
        self.client.get(f"/bookings?age={age}&date={date}", name="/bookings")

Relaunch your script and each task will be executed with a different line of the CSV file you created, making each request different (if you send more requests than there are lines in your CSV file - it will quickly happen - further requests will be sent starting over from the start of the file).

Note that here we add a name argument to our get method: Locust will group all these requests in one line in its report, which would prove very convenient for analysis!

Stay tuned: Locust is evolving

In the 2020 exhaustive comparison of load testing software (done by a k6 engineer 😉), Locust was pinpointed for its low performance and the clumsiness of running it locally on several CPU cores. Since then Locust can run a process for each of your cores with a single command:

locust --processes -1

Although Locust was created in 2011, it keeps gaining functionality, making it worth giving it a (re)-try!

Hexagonal Architecture for Dummies by a Dummy

Gatien Ducornaud — Tue, 13 Feb 2024 15:07:59 +0000

TL;DR: You’ll discover the advantages of Hexagonal Architecture in:

Simplifying Your Code: Transforming intricate coding and testing efforts into manageable, beginner-friendly tasks.
Gaining Real-World Insights: Understand how applying Hexagonal Architecture in actual projects not only upholds code quality but also promotes ongoing learning.
Plus, benefit from practical code examples I’ve tackled as a junior developer, illustrating these concepts in action!

As a rookie, Hexagonal Architecture can be your best friend 👨‍💻

I arrived on my first project with no prior experience in web development and a very academic/theoretical approach to code. It was also the tech lead's first project in such a role. However, he had experience in hexagonal architecture and believed in its power to help produce quality code, even for a rookie. It is the learnings from this experience that I wish to share here.

Hexagonal Architecture in a (non-boring) nutshell 🧠

As I understand it, hexagonal architecture aims to protect your domain (where your business logic resides) from the real world, thanks to ports. All processes and data need to go through gates (the ports), so even if some aspects outside of the domain change, if they still follow the rules set by the ports, the code in the domain does not need to change.

In practice, the ports are interfaces, and we ensure that everything goes through ports thanks to dependency inversion, where the implementation of the interfaces does not know of other implementations, but rather other ports. Thanks to that, any implementation of a given interface can be swapped out for any other implementation with no consequence to the code.

Hexagonal Architecture to the rescue of junior developers 🛟

Discovering and improving the codebase step by step ⛏️

By design, Hexagonal Architecture enforces SOLID principles, so the scope of each change can be limited to one file. Do you need to add a new route? Just modify a controller! Need to sort your data a different way? Just change the sorting in your domain! Thanks to dependency inversion, you will not have to chase your changes all over your codebase.

For example, we link users with agencies, and those “linkings” can be discarded. For some use-cases, we do not want to return the discarded linkings. To us, it is one line of code:

@Override
public List<Linking> getLinkings(UserId userId) {
    return linkingRepositoryPort
            .getLinkings(userId)
            .stream()
            .filter(linking -> !linking.discarded())
            .sorted((linking1, linking2) -> linking2.createdAt().compareTo(linking1.createdAt()))
            .toList();
}

Implementing new features is facilitated ⚡

The example given previously works fine if you are modifying existing code, but what about a brand new feature? In that case, you can copy the existing structure, as most the boilerplate code is the same. You don’t have to be familiar with Hexagonal Architecture from the start, as adapting existing pattern will (most of the time) result in a new feature that also respects Hexagonal Architecture itself.

For example, when I created the feature I presented in the previous example, I copied the existing pattern we had for Agencies and created (from the database to the route):

The table
The entity to represent the data in the table
The repository interface (with SpringJPA) to get the data from the table
The repository port interface to better use the data form my domain
The repository service implementing the port to actually call the repository to get the data
The controller to create the routes needed for the feature
The feature port to define the actions of the domain
And the service in the domain to implement the feature port

Writing tests is straightforward 🧪

I was always taught that tests were important, but I was never made to write them, so I rarely did, as it was often painful to do so. However, hexagonal architecture simplifies testing tremendously, especially for the more worthwhile domain tests, where you check if your core logic is sound.

Here is an example of one of my tests, intentionally without any context:

@Test
void should_save_linking_if_same_valid_does_not_exists() {
    // GIVEN
    doReturn(Optional.empty())
        .when(linkingRepositoryPort)
        .getValidLinkingId(USER_ID, AGENCY_ID);

    // WHEN
    linkingService.saveLinking(USER_ID, AGENCY_ID);

    // THEN
    verify(linkingRepositoryPort).saveLinking(USER_ID, AGENCY_ID);
}

@Test
void should_not_save_linking_if_there_is_already_a_valid_one() {
    // GIVEN
    doReturn(Optional.of(LINKING_ID))
        .when(linkingRepositoryPort)
        .getValidLinkingId(USER_ID, AGENCY_ID);

    // WHEN
    LinkingId result = linkingService.saveLinking(USER_ID, AGENCY_ID);

    // THEN
    assertThat(result).isEqualTo(LINKING_ID);
    verify(linkingRepositoryPort, times(0)).saveLinking(any(), any());
}

Those tests can be read even without knowledge of the code and also serve to document the intended behavior of the domain. Furthermore, I didn't have to delve into specifics on how to mock my database or set up and tear down a specific object, so tests practically write themselves!

(As a footnote: as you can also see, given Dependency Injection, you already have your Ins and Outs, so it makes it extremely easy to do Test Driven Development, and to check if every main use case of your domain is tested. I personally found it very useful to get into the habit of writing tests, and to write documentation, as the tests are the documentation)

Focusing on small tasks leading to everyday learnings 🔁

I also found Hexagonal Architecture to be a great help in promoting everyday learnings. As you can play with specific functionalities without having to worry about how they'll impact the rest of the code, you can deep dive into specific issues when they arise.

As an example, we wanted to improve the number of queries of repositories made for each request. Given that we could be certain that the resulting data would be used in the same way, I was free to experiment with the way the entities were linked

For example, in my LinkingEntity, whether I have

@ManyToOne(cascade = CascadeType.ALL, fetch = FetchType.LAZY)
@JoinColumn(name = "agency_id")
@OnDelete(action = OnDeleteAction.CASCADE)
AgencyEntity agency;

@ManyToOne(cascade = CascadeType.ALL, fetch = FetchType.EAGER)
@JoinColumn(name = "agency_id")
@OnDelete(action = OnDeleteAction.CASCADE)
AgencyEntity agency;

does not matter for the rest of the application, and I could see the number of request made in each case, and explore the impact on the repository.

However, there are still difficulties I faced and I would like to highlight so they can be anticipated.

Hexagonal Architecture drawbacks for a Junior Developer 🚧

Hexagonal Lingo can be overwhelming 📚

When I first started on the project, I had all the names mixed up. “What do you mean a Repository is not what implements a Repository Port? Why does an implementation in my domain implement an interface outside of it?”

In order to sort it out, I found it useful to visualize it by placing each type of class or package in the theoretical diagram. This was a good help to link the theoretical aspects to the way we used it in the code. For example, a typical feature would look like this in our case:

There are a LOT of patterns 🔢

When I first started to get confused about what went where, I drew a diagram similar to the previous one and thought that I understood how it worked. But then I had to do a request towards another web-service, and I had no idea how to do it. So I copied the API interface I saw for performing GET calls, but there was class implementing such interface. Turned out that the diagram looked like this:

It may not seem very different, but it does imply non-trivial differences in the code. So it implies continuous learning about hexagonal architecture itself, not a just an initial time investment.

Lots of code for simple things 💣

Following learning the new pattern for API, I implemented it for the feature. In order to do that, I created or modified the following:

a controller,
a new class for the data transfer object,
the feature port,
its implementation,
the object class in the domain,
the API,
the class for the object how it is received,
the configuration to instantiate the API.

In total, I created 6 new files and modified 2 others for what amounted basically to "Call a given service, tweak the return data a bit, and return that," which could have been done in a much shorter way.

Such work for a simple feature can be frustrating, especially it is repeated. There, the theoretical knowledge of the flexibility the hexagonal architecture offers was a good justifier of why such work was required.

Parting words

As you may guess, I would highly recommend using hexagonal architecture on your own projects. But, based on my own experience as a rookie, I would also recommend it if you are in charge of a project with junior developers, so as to help them improve their skills and produce quality code faster.

Happy hexagonal coding! 👨‍💻🌐

How to beautify java code reliably

jeremiec — Wed, 07 Feb 2024 13:50:00 +0000

I recently had to set up code formatting on a spring boot java application. Auto code formatting is important to avoiding useless diffs in source files, reducing noise in code review, allowing reviewers to focus on what matters.

Ideally we want:

Automatic formatting on file save in IDE's (vscode, IntelliJ)
Capacity to run in CI

I wanted to integrate well with vscode and after trying some solution like:

java autoformat, which I couldn't run in CI and that was not really portable between IDEs
maven auto formatting plugin (didn't run on save easily in IDEs)

I ended up, from a friend suggestion, looking into a java prettier plugin. Prettier allowed for an easy integration with most Web IDE, can be configured to format on save for vscode and is trivial to run in CI.

Installing prettier for java

Let's start with the setup in command line:

Install nodejs: (I highly recommend using a node version manager like fnm) and to install a recent node version (current long term support is 16+)
Add a package.json file at the root of your project and add prettier and the java plugin:

{
  "name": "api",
  "version": "0.0.1",
  "private": true,
  "devDependencies": {
    "prettier": "2.5.1",
    "prettier-plugin-java": "1.6.1"
  },
  "engines": {
    "node": "16.x"
  },
  "scripts": {
    "format": "prettier --write \"{,**/}*.{java,yml,yaml,md,json}\"",
    "check-format": "prettier --check \"{,**/}*.{java,yml,yaml,md,json}\"",
    "lint": "npm run check-format"
  }
}

Install the required node_module: npm install

💡 Don't forget to add node_modules to your .gitignore.

💡 I highly recommend adding a .prettierignore that list autogenerated files to avoid formatting those, eg:

node_modules
target

You can now format java code running npm run format 🚀

💡 Do not forget to add node as a dependency for your project installation and to run npm ci in your project installation script for new developers.

We now want to improve developer experience by integrating with IDEs, running the formatter in CLI being impractical.

Monorepo

If you are using a monorepo, I highly recommend setting up prettier in the root of your monorepo instead of setting up prettier in each sub directory.

Integrating with IDEs

vscode

To set up prettier in vscode: add the prettier for java extension to recommended vscode extensions for the project, to do so, you can simply add the id of the extension to: .vscode/extensions.json:

{
  "recommendations": [
    "pivotal.vscode-boot-dev-pack",
    "vscjava.vscode-java-pack",
    "redhat.vscode-xml",
    "sonarsource.sonarlint-vscode",
    "gabrielbb.vscode-lombok",
    "dbaeumer.vscode-eslint",
+   "esbenp.prettier-vscode", // the classic prettier extension
    "shengchen.vscode-checkstyle",
  ]
}

then set up this extension as the default linter for java file in .vscode/settings.json :

{
// rest of your configuration
// automatic format on save, I recommend the save on focus lost option as well
+   "editor.formatOnSave": true,
+   "[java]": {
+       "editor.defaultFormatter": "esbenp.prettier-vscode"
+   }
}

Already we can easily format java code on save:

I highly recommend checking in the .vscode folder in your versioning tool (most likely git) to share this configuration with all developers.

IntelliJ

To integrate with prettier java formatter:

First install the File Watchers IntelliJ plugin
Then configure the plugin Tools/File Watchers:

Name: Prettier-java
File type: Java
Program:  full path to `.bin/prettier` eg: `node_modules/.bin/prettier`
Arguments: --write $FilePathRelativeToProjectRoot$
Output paths to refresh: $FilePathRelativeToProjectRoot
Auto-save edited files to trigger the watcher: check it
Trigger the watcher on external changes: check it

prettier documentation can be found here for IntelliJ integration

Integrate code formatting check in CI

Now we only need to integrate this check in CI, I'm using gitlab for this example, but the principles can be transposed to any other CI.

First we need to add a new job:

stages:
  - build
  - test
  - deploy

# ... rest of the file

# beautify backend files with prettier
lint-back:
  image: node:lts-alpine # We use a node image to run prettier
  stage: test # must match an existing stage
  needs: [] # none required optimization that explain to gitlab ci that this test has no dependency and can start as soon as possible
  script:
    - npm ci
    - npm run lint
# ... rest of the file

now just push your change, and you are all set:

Whisper to your keyboard: Setting up a speech-to-text button

jeremiec — Tue, 06 Feb 2024 13:58:00 +0000

Long story short: I broke my arm while riding my bike and I can't type. Quite the problem as typing is a key part of my daily life as a software engineer. So I decided to add a speech to text button to my keyboard. Here's how I did it.

How to transcript speech to text on linux

First, I looked into how to transcript speech to text on linux. I found a few solutions:

Whisper.cpp: a way to run the OpenAI whisper model locally
OpenAI's API: the original Whisper model as a pay as you go API
Deepgram: a pay as you go service that offers a speech to text API

After testing whisper.cpp on my machine, it was too slow and inaccurate (out of the box), so I decided to use an API and abandoned the idea of running it locally (for now).

So I registered for an OpenAI account and tried their API:

(you can download an example recording here)

FILEPATH="./recording" # path to the recording without extension

curl --request POST \
  --url https://api.openai.com/v1/audio/transcriptions \
  --header "Authorization: Bearer $OPEN_AI_TOKEN" \
  --header 'Content-Type: multipart/form-data' \
  --form file="@$FILEPATH.wav" \
  --form model=whisper-1 \
  --form response_format=text \
  -o "${FILEPATH}.txt"

It's quite magical, already I could transcript my voice to text.

How to record your microphone on linux

This one was a challenge for compatibility and device selection, but basically, what you want to do is record your microphone as a .wav file and save it. (I could not make mp3 encoding work reliably and it was not required)

First use arecord to list your available devices:

arecord -l

Then you should be able to test which input device is your microphone by recording a few samples:

FILEPATH="./recording"
AUDIO_INPUT="hw:0,0" # your microphone device, test with a few devices to find the right one
arecord --device="$AUDIO_INPUT" --format cd "$FILEPATH.wav" --duration=10

How to write the text file emulating a keyboard

This one is quite easy, I used xdotool to emulate a keyboard and write the text file:

FILEPATH="./recording"
perl -pi -e 'chomp if eof' "$FILEPATH.txt" # remove trailing newline if any to avoid sending an extra newline keypress
xdotool type --clearmodifiers --file "$FILEPATH.txt"

Putting it all together: a button to transcript speech to text

Now that we have all the pieces, we can put it all together in a script:

#!/usr/bin/env bash
# usage: exec ./voice-typing.sh twice to start and stop recording
# Dependencies: curl, jq, arecord, xdotool, killall

set -euo pipefail
IFS=$'\n\t'

# Configuration
readonly PID_FILE="${HOME}/.recordpid"
readonly FILE="${HOME}/.voice-type/recording"
readonly MAX_DURATION=15
readonly AUDIO_INPUT='hw:0,0' # Use `arecord -l` to list available devices

start_recording() {
  mkdir -p "$(dirname "$FILE")"
  echo "Starting new recording..."
  nohup arecord --device="$AUDIO_INPUT" --format cd "$FILE.wav" --duration="$MAX_DURATION" &>/dev/null &
  echo $! >"$PID_FILE"
}

stop_recording() {
  echo "Stopping recording..."
  if [ -s "$PID_FILE" ]; then
    local pid
    pid=$(<"$PID_FILE")
    kill "$pid" && wait "$pid" 2>/dev/null || killall -w arecord
    rm -f "$PID_FILE"
    return 0
  fi
  echo "No recording process found."

}

write_transcript() {
  perl -pi -e 'chomp if eof' "$FILE.txt"
  xdotool type --clearmodifiers --file "$FILE.txt"
}

transcribe_with_openai() {
  curl --silent --fail --request POST \
    --url https://api.openai.com/v1/audio/transcriptions \
    --header "Authorization: Bearer $OPEN_AI_TOKEN" \
    --header 'Content-Type: multipart/form-data' \
    --form file="@$FILE.wav" \
    --form model=whisper-1 \
    --form response_format=text \
    -o "${FILE}.txt"
}

main() {
  if [[ -f "$PID_FILE" ]]; then
    stop_recording
    transcribe_with_openai
    write_transcript
  else
    start_recording
  fi
}

main

The script needs to be run twice to start and stop recording. It will then transcript the recording and write the text to the current window.

To trigger it with a button, simply add a keyboard shortcut to run the script in your keyboard configurations settings on your linux distribution.

Conclusion and next steps

It was a fun little project to do, and it's quite useful to be able to type with your voice. I'm not sure I'll keep using it after my arm heals, but it's a nice option to have.

Since using whisper was a little slow, I tried and switched to Deepgram for faster (and sometimes more accurate) transcriptions. I published the complete script on github, you can find it here, it checks for requirements and handles errors a little more gracefully.

Effective nodejs version management for the busy developer

jeremiec — Sun, 04 Feb 2024 12:56:50 +0000

I highly recommend setting up nodejs with a version manager, nvm was and still is a popular option, however, I now recommend and have been using fnm, a simpler and faster alternative to manage my nodejs versions.

Install a nodejs version manager: FNM with automatic version switching

To install fnm, follow the installation steps.

I highly recommend installing the autocompletion, if using zshrc, add this in your ~/.zshrc:

# if fnm is installed, add to path and load autocomplete
if [ -d ~/.fnm ]; then
  path+=~/.fnm
  # setup env and allow for automatic node version change when directories contains `.node-version` or `.nvmrc`
  eval "$(fnm env --shell=zsh --use-on-cd)"
  # shell completion
  source <(fnm completions --shell zsh) > /dev/null 2>&1
  compdef _fnm fnm
fi

Restart your shell to take those changes into account: exec $SHELL -l

Use a nodejs version

Then you can easily install and switch node versions:

node -v
# install a version
fnm install lts/iron
# switch to version
fnm use lts/iron
# you can switch to a set version by number as well (eg. v20.9.0)
node -v

Project setup to promote the correct nodejs version

When using node in a project, I recommend:

Setting up a .nvmrc file at the root of your project to declare the used nodejs version:

lts/iron

Using the same version in your runtime, CI, for example with docker:

FROM node:iron-alpine

In your project package.json, I recommend using the engines field to declare the nodejs version:

{
    "engines": {
        "node": ">=20",
        "pnpm": ">=8"
    }
}

as well as using engine-strict in your .npmrc file:

engine-strict=true

Another option is using a preinstall hook for pnpm.

Bonus using pnpm as a package manager

I also recommend using pnpm as a package manager, it's faster and more efficient than npm or yarn with great capabilities concerning monorepo setup. On recent nodejs versions (v16.13+), you can install it easily with:

corepack enable && corepack prepare pnpm@latest --activate

use engines to enforce usage of pnpm or a preinstall hook:

{
    "engines": {
        "pnpm": ">=8"
    }
}

{
  scripts: {
    "preinstall": "npx only-allow pnpm",
  }
}

How to boost your database performances under JPA part 1: @Index 📖

Hugo Brunet — Fri, 19 Jan 2024 14:30:00 +0000

TL;DR: I will present you how the use of an ORM and its given tools - here @Index - can boost your application’s performance thanks to the illustration of a booming business and its scale related issues 🚀

Introduction to Object-Relational Mapping (ORM) with JPA

Databases play a crucial role in most applications. However, working directly with databases can be complex and time-consuming. This is where Object-Relational Mapping frameworks like Java Persistence API (JPA) come into play.

ORM frameworks provide a way to map Java objects to relational database entities, allowing developers to interact with databases using object-oriented programming concepts. JPA, is a popular ORM framework that simplifies database operations and enhances productivity.

Your first trusty partner: JPA 👨‍💼

As the CEO of a new company, you'll soon find yourself needing to retrieve data from a database. But don't worry, you don't have to deal with the difficulty of writing SQL queries and managing connections yourself. That's where our trusty associate, JPA, comes in. With JPA, you can simply ask it to fetch the data you need, and it will take care of all the dirty work for you 😉

But before searching for employees, you need to have some so here is how we will define them:

@Entity
@Table(name = "employees")
public class Employee {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "name")
    private String name;

    @Column(name = "age")
    private int age;

    @Column(name = "departmentCode")
    private int departmentCode;

    // ... additional properties, getters, and setters
}

Let's say you want to retrieve a list of employees based on their department code. Without JPA, you would have to manually write the SQL query, manage the database connection, and map the query results to Java objects 😪

But with JPA, here is what you can do:

import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;

@Repository
public interface EmployeeRepository extends JpaRepository<Employee, Long> {

    List<Employee> findByDepartmentCode(int departmentCode);
}

Your only need is to use the EmployeeRepository interface, which is already set up for you. Just call the findByDepartmentCode method, passing in the department code you're interested in, and JPA will handle the rest. It will execute the query, map the results to Java objects, and manage the database connection behind the scenes 🤩

So, sit back and relax while JPA does the heavy lifting for you. You can focus on running your company while JPA takes care of retrieving the data you need from the database 🏝️

Handle your database scale with `@Index` annotation 📈

2 years later, the company has experienced tremendous growth, and thousands of employees have been hired. However, the trusty associate, JPA, is struggling to keep up with the increasing workload, even finding the list of employees related to a department takes hours. As the CEO, you are determined to find a solution to help JPA handle the company's growth effectively.

Fortunately, you come across a powerful tool called the @Index annotation. This annotation is designed to optimize query performance by creating indexes on specific columns of the database tables and put some order in JPA’s files.

This guy won’t believe his eyes when he’ll see his room so clean thanks to this single line of code:

@Entity
@Table(
    name = "employees",
  **indexes = { @Index(name = "department_code_idx", columnList = "departmentCode") }**
)
public class Employee {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "name")
    private String name;

    @Column(name = "age")
    private int age;

    @Column(name = "departmentCode")
    private int departmentCode;

    // ... additional properties, getters, and setters
}

All you have done by putting this index in @Table is build an ordered structure outside the database in which you can easily find a departmentCode. Why department codes and not age ? Because I said columnList=”departmentCode”, nothing more. So when JPA will find for datas in a specific department, the search will be blazingly fast ⚡

⚠️ Keep in mind that you are maybe going to build some more indexes so even if it’s optional, name them well as I said with name = "department_code_idx".
You better not add a new department everyday or you will have to destroy and re build this data structure because order may have changed ⚠️

If you don’t want your employees to socialize and would rather only allow one unique employee per department, you can set your index to force unicity of departmentCode thanks to the last available parameter: unique

@Index(name = "department_code_idx", columnList = "departmentCode", **unique = true**)

As the CEO, you can now focus on running the company, confident that JPA, with the help of the @Index annotation, can effectively handle the data management needs of the expanding organization.

Let’s level up: how to index a join table column in JPA ?

You are now a direct concurrent of the GAFAM, thus your employees will need to work on many different missions at the same time and JPA will need to find them in the blink of an eye. Previously, JPA went into the model of the employee to add an index, which means he needs access to the the table in order to index, what if JPA wants to index a table which he can’t instantly access ?

ManyToMany relations bring this kind of situation where a new table is created without being self suffisant, these are Join Tables and are created as shown:

@Entity
@Table(name = "employees")
public class Employee {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "name")
    private String name;

    @Column(name = "age")
    private int age;

    **@ManyToMany
    @JoinTable(name = "employee_mission",
            joinColumns = @JoinColumn(name = "employee_id"),
            inverseJoinColumns = @JoinColumn(name = "mission_id"))
    private List<Mission> missions;**

    // ... additional properties, getters, and setters
}

@Entity
@Table(name = "missions")
public class Mission {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "name")
    private String name;

    // ... additional properties, getters, and setters
}

Fortunately @JoinTable annotation allows us to pass one more parameter: indexes which takes a list of indexes to add in the table, so if i want to index both of my columns here is what I can do

  @ManyToMany
    @JoinTable(name = "employee_mission",
          joinColumns = @JoinColumn(name = "employee_id"),
          inverseJoinColumns = @JoinColumn(name = "mission_id"),
                indexes = { 
                        @Index(name = "employee_id_idx", columnList = "employee_id"),
                        @Index(name = "mission_id_idx", columnList = "mission_id")
                }
    )
  private List<Mission> missions;

Now, a new structure is built in JPA’s office which allows him to quickly find all the missions affected to a given employee and all employees working on a given mission.

I wish you a good luck for the rest of your journey with this successful business with JPA and his new favorite tool @Index 😎

`@Index` in a nutshell

The @Index annotation in JPA is used to define indexes on specific columns of database tables. It has three parameters:

columnList: Specifies the name of the column(s) on which the index should be created.
name: (Optional) Specifies the name of the index.
unique: (Optional) Specifies whether the index should enforce uniqueness on the column(s) it is created on.

@Index can be used in various JPA annotations, such as @Table and @JoinTable or by itself.

The use of @Index has divided the response time of some routes in my project by at most 42 with a route going from 26.51s to 0.634s.

Stay tuned for the next parts of this series, where we will explore other techniques to boost database performances under JPA.

How to ensure that all the routes on my Symfony app have access control

Thomas Hercule — Wed, 17 Jan 2024 10:15:05 +0000

TL;DR

Effective access control involves both Symfony's firewall and specific control on each route.
If you use API Platform, use ACCENT to verify access control on your routes.
If you use functions in the controller to secure your routes, you can install the composer package I created based on the following part of this article.
If you're not using API Platform nor functions, you can create a custom script.
It's important to integrate access control verification into your CI process to maintain a high level of security.

What is access control

Access control allows you to define access permissions to specific parts of your application. It helps restrict access to certain pages or features for users who do not have the necessary permissions.

To implement access control, you need to define user roles and corresponding permissions, then apply them to the routes of your application. This can be especially useful for safeguarding sensitive information or important actions, such as modifying or deleting data.

Implementing access control for routes significantly enhances the security of your Symfony project and safeguards your users’ data.

Effective access control in your Symfony project involves two main aspects:

Symfony Firewall
Specific access control for each route

Symfony firewall

The Symfony firewall is the initial layer of security for routes, adding global rules to all routes or specific groups of routes.

The configuration for this firewall is typically found in config/packages/security.yaml.

In this file, you can:

Define which URL groups require (or do not require) security checks (logged-in user, specific role, GET/POST requests, etc.).
Whitelist IPs for certain endpoints.

Generally, you would grant access to the site only to logged-in users (the default behavior) and disable this check for specific pages (login, password reset, etc.).

Here's an example of a typical security.yaml configuration where we ensure the user is logged in for all routes except the login and register pages.

access_control:
  - { path: ^/login, role: IS_AUTHENTICATED_ANONYMOUSLY }
  - { path: ^/register, role: IS_AUTHENTICATED_ANONYMOUSLY }
  - { path: ^/, roles: [ROLE_USER, ROLE_ADMIN] }

The role IS_AUTHENTICATED_ANONYMOUSLY allows access to pages even for non-authenticated users. You can find more details in the Symfony firewall documentation.

Although it's possible to have role-based access control in the firewall, it's preferable to implement it on each route:

To handle more complex rules.
To make the access control directly visible on the relevant functions.

If you have a Symfony project with more than a dozen routes, as it was the case on my project, you will need a tool to automatically ensure they all have access control.

How to ensure access control for all your symfony routes

Depending on the type of project, different solutions can be used to verify the security of these routes. If you are using API Platform, you can save time by utilizing ACCENT. Otherwise, it is possible to create a custom script since there is no pre-existing tool to automate this verification.

My project uses API Platform: using ACCENT

If you use API Platform, you can use ACCENT. It is a powerful tool to generate a report on the access control of each of your routes with very little configuration.

To install ACCENT, run: composer require --dev theodo/accent-bundle.
Then, you can generate a detailed report on the access control of your routes with: bin/console theodo:access-control

Example of a generated report:

Above you can see the list of all the projects’ routes with, highlighted in red, the routes that lacks access control.

My project doesn't use API Platform

My project uses functions to secure routes

If your project uses functions to secure your routes in the controllers as such :

class AdminController extends AbstractController
{
    public function createUser(Request $request) {
            if (!$securityService->is('admin')) {
                // We redirect the user to the login page
            }
            // ...
        }
}

It also works with Symfony's function (isGranted and denyAccessUnlessGranted for example).

You can use the composer package I created based on the script I build in the next part of this article : symfony-route-security-checker.

Install it with : composer require --dev th0masso/symfony-route-security-checker.
Copy the ssacc-config.dist.yaml from the package repository into your project. Read the documentation to configure it.
Finally, you can generate a detailed report on the access control of your routes with: bin/console security:check-routes

Other cases

You can create a custom script specific to your project. In this section, we will use an example of a script that checks if Symfony security functions are called at the beginning of each function related to routes. If you are using something other than Symfony Security function, this section can still help you write such a script.

Plan for this section:

Access control with Symfony Security
Why write a script?
Retrieving the list of project routes
Verifying that the function has access control

Access control with Symfony security

There are several ways to secure routes "manually" using Symfony Security functions: isGranted and denyAccessUnlessGranted.

In annotations:

use Symfony\Component\Security\Http\Attribute\IsGranted;

class AdminController extends AbstractController
{
        #[IsGranted('ROLE_ADMIN')]
    public function createUser(Request $request) {
            // ...
        }
}

Or by directly calling these functions:

use Symfony\Component\Security\Http\Attribute\IsGranted;

class AdminController extends AbstractController
{
    public function createUser(Request $request) {
            if (!$this->isGranted('ROLE_ADMIN')) {
                // We redirect the user to the login page
            }
            // ...
        }
}

I have worked on a project where we controlled access to our routes by calling isGranted or denyAccessUnlessGranted within our functions. However, the method I used to verify access control on our routes can also work with annotations with minor modifications.

Why write a script?

As there is no existing tool to automatically verify routes in these cases, we have two solutions to ensure that routes have specific access control:

Manually check the routes.
Create a script to do it for us.

Both of these solutions provide a snapshot of the current state. However, creating a script automates the access control verification for future routes and could potentially be shared and used in other projects.

In my case, I knew there were over a hundred routes in the project, so doing it manually would have been very redundant.

Retrieving the list of project routes

Symfony provides a command to list all the routes in the project along with their paths.

php bin/console debug:router --show-controllers --format=json

Now, for each route in the project, we have its path in the format: MyPath/MyController.php::MyFunction

Verifying that the function has access control

Now that we know which functions are associated with which routes and their paths, we can check in the files whether these functions indeed have calls to permission-checking functions such as isGranted or denyAccessUnlessGranted.
These functions can be called through annotations or directly within the controller function.

Using annotations to restrict access

If you use annotations to secure your routes, you can:

recover the annotation of the controller's function as a string with PHP's ReflectionProperty::getDocComment
then search for isGranted or denyAccessUnlessGranted using a regular expression.

Since I didn't use annotations on my project, I will not go into details on this method.

Calling functions to restrict access

If you call some permission-checking functions directly into your controller, it's possible to ensure that these functions are called at the beginning of the function used by the route in two ways:

Using a regular expression
Using the Abstract Syntax Tree (AST)

AST should be more robust because it can handle more complex cases. For example, with AST, we can determine if the function is called in a condition, a loop, or another function. However, within my team, no one had experience with AST manipulation, and we didn't encounter such complex cases in my project. So, I opted for a solution using a regular expression.

I then wrote a regular expression that matches !$this->isGranted or $this->denyAccessUnlessGranted on the first line of the function used by the route (⚠️ please do not read this monstrosity):

$regex = '((public function ' . $routeFunction . ')(\([^{]*\{)(\s.*)(\$this->denyAccessUnlessGranted|!\$this->isGranted))';

Now, it's just a matter of checking if we find the expression in the file.

preg_match($regex, $fileContent)

It worked on the first try; all the routes were secured.

I’m kidding, I'm not fluent in regular expressions, and I want others to be able to read and understand this script, so it took me a few hours.

ℹ️ Fortunately, I was advised to use regex101.com, an incredibly useful website for testing and understanding complex regular expressions.

Then, I broke down and commented the expression to make it more understandable:

// start of regular expression
$regex = '(';
// find "public function myFunction"
$regex .= '(public function ' . $function . ')';
// then everything until "{"
$regex .= '(\([^{]*\{)';
// then everything on next line
$regex .= '(\s.*)';
// until "$this->denyAccessUnlessGranted" or "!$this->isGranted"
$regex .= '(\$this->denyAccessUnlessGranted|!\$this->isGranted)';
// end of regular expression
$regex .= ')';

Next, I ran my script with this final version of the expression.

ℹ️ You can find the composer package based on this script on github.

The file with the regex and the rest of the logic is on the same repo : src/Command/AccessControlCheckerCommand.php.

Security vulnerabilities found on my project

46 Routes Without Access Control ?

After investigation:

7 routes linked to Symfony modules: web_profiler and twig
6 routes that should not be verified, therefore accessible to everyone (login page, password recovery, etc.)
4 exposed API routes, which should not be verified in this way
9 false positives due to special cases
3 unused routes (cleaning up dead code 🤩)
17 routes that were indeed problematic

After identifying where each of these 17 routes was being used on the site, I realized the importance of having a strict access control strategy for your app.

For these routes without permission checks, most were harmless, but some allowed critical actions on the site! This means that any logged-in user could perform these actions if they sent the right HTTP request (thanks to Symfony's firewall rejecting non-logged-in users). However, there was no risk of privilege escalation.

To secure these 17 routes, I organized a meeting with the client to define the access control to be applied for each one.

Afterwards, the client lived happily ever after, and the site wasn't hacked due to an access control issue... until the day a developer introduced a new route without permission!

How to ensure new routes are secure too

To ensure that new routes added are also secure, it's important to run this verification script automatically. The best way is to add it to your CI (Continuous Integration) to ensure that the script runs before each merge. If you can't add it to your CI for some reason, you can add it to your git pre-commit or pre-push hook it's not as good because those hooks can be ignored by the developer by using --no-verify after git commit or git push.
Unfortunately, I didn't have the time to implement it in my project's CI due to time constraints.

Although the performance of my script is quite good, it still takes a few seconds to run if you have hundreds of routes (approximately 5 seconds for 200 routes). To overcome this issue, one approach could be to run the script only on modified files, which would significantly reduce processing time.