The problem with maintaining code quality in a large technology company; and the solution.

Tzach Bonfil — Thu, 28 Jul 2022 06:13:00 +0000

We've all been there, from emails to spam "@here" in Slack. When we have a change in our tools that needs to be updated by another team, this process can become a frustrating task.
This example and many more are day-to-day issues that big companies face.
If you have created tools or integration that other teams use, you probably had to make an update or a change, and sometimes these changes cannot wait. The usual play is to send a large distributed email requesting to update your dependency.
At best, it would go to their backlog, and at worse, it will be ignored.
At Autodesk, we understood this process should be automated.

Our story started with a simple package named "ProjectPageLayout".
All Autodesk ACS teams (20+) used this package, and once we wanted to make a change, we had to request every team to update it to get a consistent experience in our application.

When we retroed this process, an idea came up, why can't we just test their "project" and make sure they use the latest version? And if it's not the latest version, we can enforce it.
This is how the idea for "ProjectLinter" came to life.

Our inspiration is ESLint. If ESLint statically analyzes your code, we statically analyze your whole project to find problems quickly. It can fit all project types, and you can run it as part of your continuous integration pipeline and send notifications asynchronously.
Sometimes it can also automatically fix your problems.

Every team can create a rule (like in ESLint) to enforce it in other projects. To make it simple for the project owners, they do not have to be familiar with the rules and specificities because when creating a rule, it is deployed and automatically executed alongside the other rules seamlessly.

In other words, it is seamless to the users of the linter and simple to create a rule for the enforcing team.

But wait, why should we limit ourselves to updating the team's dependencies and business logic rules if we can also ensure their projects possess the best standards and quality in the industry.
We also create rules that ensure performance, minimal bundle sizes, scan security vulnerabilities, high testing coverage, and minimal package duplication.

Another massive win from this process is that it can give us a way to measure infrastructure tool's success. We can define KPIs according to rules, for example, “Performance” (Web vitals).
We can see changes over time for this rule and therefore understand whether there is a wide problem across teams that requires infrastructural assistance and whether this assistance is successful.

And then, a problem was raised that sometimes teams maintain code bases that do not change frequently, and if we wait for a code change to trigger the linter, it could be too late.
The solution is not just to run the linter when the codebase is changed but also over time, for example, daily and asynchronously.

Another issue is that with all due respect to the importance of the rules, you can not break the team's process out of nowhere.
Therefore we created gradual diagnostics systems.
You can set that currently a rule would just warn the teams and send them notifications, but after two months of no response, it would take more extreme measures.

Let me close by saying: I genuinely believe this is another step toward a simpler future with more automation. The beautiful thing is that my team and I had the opportunity to work on that during the hackweek at Autodesk TLV on an idea
we are passionate about and love.

My Teammates:

What happens inside the CSS Webpack Plugin - Style the Modern Web

Tzach Bonfil — Sun, 13 Mar 2022 14:44:28 +0000

Styling a modern application is no simple task - traditionally it is done by serving HTML with CSS for styling, while sprinkling in some JavaScript to get the job done.

How do you modernize this approach of setting up an app? We might think we know the answer - to use a bundler like Webpack and a JavaScript framework / library like React.

But how do we handle the CSS, and why isn’t it as simple as you would expect?

Agenda -

Part 1: Understanding the issue with native CSS.
Part 2: Setting up our Webpack application without a CSS plugin.
Part 3: Writing the Loader.
Part 4: Writing an advanced Plugin.

If you are here just for implementation information, skip to part 3.

Disclaimer - This is not a production-ready plugin. To see one that is, check out a project my team and I are working on - Stylable.

Part 1: Understanding the issue with native CSS.

Our options

Native CSS is implemented in different ways:

The first (and the simplest) way to include CSS is using inline styling, which means that you explicitly include a style in an HTML tag. <span style="color:red;">...</span>
Another solution is to use an HTML tag called <style>...</style>, where its text content is the style itself, and it is used to target the different HTML elements.
And yet another option is to load a CSS file via a link tag and target the different HTML elements inside that file.

The problems

Each of the solutions above has its benefits and tradeoffs. It is very important to understand them to avoid unexpected behavior in your styling. You'll find that none of those solutions, however, solve one of the most problematic issues - that CSS is global.

The global issue is a pretty difficult one to overcome. Let's say you have a button with a class called btn and you style it. One day your co-worker works on a different page that has a button too, and he also decided to call it btn. The problem should be apparent - the styles would clash.

Another significant issue is specificity, where the specificity is equal between selectors, and the last declaration found in the CSS is applied to the element. To put it simply - your order matters.

Part 2: Setting up our Webpack application without a CSS plugin.

The solutions

Currently, there are many different solutions to these problems, from utility frameworks, CSS preprocessors, and other things that all try to help with the issues that native CSS has.

In this article, I would like to solve some of those problems from scratch with you.

First, let's set up our environment real quick. To do this, run these commands:

(We create a directory, initializing our package.json, and installing Webpack, and Babel dependencies)

mkdir example-css-plugin
cd example-css-plugin
npm init -y
npm i -D webpack webpack-cli @webpack-cli/generators @babel/preset-react
npm i react react-dom

When the development dependencies have finished installing, run the Webpack init command:

npx webpack init

For our setup, your answers should look like this:

? Which of the following JS solutions do you want to use? ES6
? Do you want to use webpack-dev-server? Yes
? Do you want to simplify the creation of HTML files for your bundle? Yes
? Do you want to add PWA support? No
? Which of the following CSS solutions do you want to use? none
? Do you like to install prettier to format generated configuration? No

Make sure that this question is answered as such: "Which of the following CSS solutions do you want to use" - none.

Configure React

Go to .babelrc and make sure that the presets array includes "@babel/preset-react".

This is not a must but it’s to make sure that our project can transform jsx.

{
    "plugins": ["@babel/syntax-dynamic-import"],
    "presets": [
        [
            "@babel/preset-env",
            {
                "modules": false
            }
        ],
            "@babel/preset-react"
    ]
}

Now we need to go to index.html and make sure it has the div with the id of “root”.

<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8" />
        <title>CSS Webpack Plugin example</title>
    </head>
    <body>
        <div id="root"></div>
    </body>    
</html>

After all that, we are ready to write our app inside src/index.js:

import React from 'react';
import { render } from "react-dom";

render(
  <div>
    Hello World!
  </div>,
  document.getElementById('root')
)

Part 3: Writing the Loader

So what are we aiming for? First things first, we want to simply load our CSS from our JS.
Let's create our CSS file and call it index.css.

.app {
    background: red;
}

And of course, use it in the index.js file:

import React from 'react';
import { render } from 'react-dom';
import './index.css'

render(
  <div className="app"> Hello World! </div>,
  document.getElementById('root')
);

Run our application:

npm run serve

Now you probably see this error in the console:

This error makes a lot of sense, as Webpack does not know how to handle CSS imports - we need to tell it how to do it.

Creating a Webpack Loader

What are loaders?

Webpack enables the use of loaders to preprocess files. This allows you to bundle any static resource way beyond JavaScript.
To put it simply, in our case, they are functions that take the CSS file as input and output a js file.
CSS -> JS

Loader implementation

Let's create a file alongside the webpack.config.js named loader.js.
Our goal is to append the style value we get from the CSS file inside the dom.
loader.js:

// Appending the style inside the head
function appendStyle(value) {
    const style = document.createElement('style');
    style.textContent = value;
    document.head.appendChild(style);
}

// Make sure it is not an arrow function since we will need the `this` context of webpack
function loader(fileValue) {
  // We stringify the appendStyle method and creating a file that will be invoked with the css file value in the runtime
  return `
    (${appendStyle.toString()})(${JSON.stringify(fileValue)})
  `
}

module.exports = loader;

Now we need to register it inside the webpack config.
webpack.config.js:

const config = {
  //... rest of the config
    module: {
        rules: [
          // ... other rules not related to CSS
            {
                test: /\.css$/,
                loader: require.resolve('./loader')
            }
        ]
    }
  // ...
}

Restart the terminal, and we got it! 🎊

What’s happening behind the scenes?

Webpack sees your CSS import inside index.js. It looks for a loader and gives it the JavaScript value we want to evaluate in runtime.

Overcoming the global issue

Now we have our style, but everything is global. Every other language solves the global issue with scoping or namespacing. CSS, of course, is not a programming language per se, but the argument still holds.
We will implement the namespace solution. This is going to give us scoping, and each file is going to have its own namespace.
For example, our import is going to look like this:

AppComponent123__myClass

If another component has the same class name, it won't matter behind the scenes since the namespace will be different.
Let's go the loader.js and add the following method:

const crypto = require('crypto');

/**
 * The name is the class we are going to scope, and the file path is the value we are going to use for namespacing.
 * 
 * The third argument is the classes, a map that points the old name to the new one.
 */
function scope(name, filepath, classes) {
  name = name.slice(1); // Remove the dot from the name.
  const hash = crypto.createHash('sha1'); // Use sha1 algorithm.
  hash.write(filepath); // Hash the filepath.

  const namespace = hash.digest('hex').slice(0, 6); // Get the hashed filepath.
  const newName = `s${namespace}__${name}`;

  classes[name] = newName; // Save the old and the new classes.

  return `.${newName}`
}

After we are done scoping the class, let's return the loader method.
We need a way to connect the scoped class selector to the user's javascript code.

function loader(fileValue) {
  const classes = {}; // Map that points the old name to the new one.
  const classRegex = /(\.([a-zA-Z_-]{1}[\w-_]+))/g; // Naive regex to match everything that starts with a dot.
  const scopedFileValue = fileValue.replace(classRegex, (name) => scope(name, this.resourcePath, classes)); // Replace the old class with the new one and add it to the classes object

 // Change the fileValue to scopedFileValue and export the classes.
  return `
    (${appendStyle.toString()})(${JSON.stringify(scopedFileValue)})

    export default ${JSON.stringify(classes)}
  ` // Export allows the user to use it in their javascript code
}

In the index.js, we can now use it as an object:

import React from 'react';
import { render } from "react-dom";
import classes from './index.css'; // Import the classes object.

render(
  <div className={classes.app /* Use the app class  */}>
    Hello World
  </div>,
  document.getElementById('root')
)

Now it works with the namespaced selector 🎉
Class with namespaced selector
Some important points about the changes we implemented.

When the loader is used by Webpack, the context will be the loader context (this) from Webpack. You can read more about it here. It provides the resolved file path, which makes the namespace unique to the file.
The way we extract the classes selectors from the CSS file is a naive implementation that isn't taking into account other use cases. The ideal way is to use a CSS parser.
this.resourcePath refers to the local path, which means that in other machines, the path may look different.

The loader is now implemented, and we've got scoped classes at this point. However, everything is loaded from JavaScript, and so it is not yet possible to cache the CSS.

To do this, we will need to compose all the CSS into one file, and to do that, we will need to create a Webpack plugin.

Part 4: Writing an advanced Plugin

As mentioned before, we implemented a loader that can inject CSS into our page. What if we want to do it with a singal file and not an injection, however?

Loading CSS as a file comes with many benefits, and the best of them is caching. A browser can cache that file and won't need to redownload it every time it is needed.

This operation is more complicated than the loader case since we will have more context on the Webpack bundling process.

What is a plugin?

A Webpack plugin is a JavaScript object that has an apply method. This apply method is called by the Webpack compiler, giving it access to the entire compilation lifecycle.

Creating the Plugin

Let's create a file called plugin.js, and create the plugin skeleton:


class CSSPlugin {
  cssMap = new Map() // We will save the CSS content here

  /**
   * Hook into the compiler
   * @param {import('webpack').Compiler} compiler 
   */
  apply(compiler) { }
}

module.exports = {
  CSSPlugin
}

Now let's implement the apply method:


class CSSPlugin {
  cssMap = new Map() // We will save the CSS content here

  /**
   * Hook into the compiler
   * @param {import('webpack').Compiler} compiler 
   */
  apply(compiler) {

    // Hook into the global compilation.
    compiler.hooks.thisCompilation.tap('CSSPlugin', (compilation) => {

      // Hook into the loader to save the CSS content.
      compiler.webpack.NormalModule.getCompilationHooks(compilation).loader.tap(
        'CSSPlugin',
        (context, module) => {

          // Setting up a method on the loader context that we will use inside the loader.
          context.setOutputCSS = (css) => {

            // the key is the resource path, and the CSS is the actual content.
            this.cssMap.set(module.resource, css)
          }
        }
      )
    })
   }
}

We hooked into the global compilation and then hooked into the loader (which was implemented previously).

When the loader content is reachable, we add the setOutputCSS method to call it from the loader.

Here's how to call this method in loader.js:

function loader(fileValue) {
  const classes = {}; // Map that points the old name to the new one.
  const classRegex = /(\.([a-zA-Z_-]{1}[\w-_]+))/g; // Naive regex to match everything that starts with a dot.
  const scopedFileValue = fileValue.replace(classRegex, (name) => scope(name, this.resourcePath, classes)); // Replace the old class with the new one and add it to the classes object

  this.setOutputCSS(scopedFileValue) // Pass the scoped CSS output

 // Export the classes.
  return `export default ${JSON.stringify(classes)}`
}

As you can see, we are not appending the style in the JavaScript. We use the method we added to the context.

After collecting all the scoped CSS content, we now need to hook into the asset process hook to let the compiler know that we have a new asset that it should handle.

Let's add it to the apply method:

class CSSPlugin {
  // ...

  apply(compiler) {
      compiler.hooks.thisCompilation.tap(
        'CSSPlugin', 
        (compilation) => {
        // ...

        // Hook into the process assets hook
        compilation.hooks.processAssets.tap(
          {
            name: 'CSSPlugin',
            stage: compiler.webpack.Compilation.PROCESS_ASSETS_STAGE_DERIVED
          },
          () => {

                // Loop over the CSS content and add it to the content variable
                let content = '';
                for (const [path, css] of this.cssMap) {
                  content += `/* ${path} */\n${css}\n`;
                }

                // Append the asset to the entries.
                for (const [name, entry] of compilation.entrypoints) {
                  assetName = `${name}.css`;
                  entry.getEntrypointChunk().files.add(assetName);
                }

                // Create the source instance with the content.
                const asset = new compiler.webpack.sources.RawSource(content, false);

                // Add it to the compilation
                compilation.emitAsset(assetName, asset);
          }
      )
  }
}

Now we'll run the build command:

npm run build

We should see main.css in the output folder, and also injected into the HTML:

Output:

index.html:

And that's it!
We finished the Plugin and have one CSS file for all of the CSS.

Note that we skipped dependencies, graph ordering, and filtering unused CSS for demonstration purposes.

You can see my full implementation with typescript and tests in this repo here.

If you have any questions you can reach me via LinkedIn. I hope I managed to help you.

DEV Community: Tzach Bonfil