DEV Community: Domenik Reitzner

Nuxt3 LCP Advanced optimizations with pnpm

Domenik Reitzner — Fri, 04 Oct 2024 07:02:38 +0000

This is just a quick one (I promise). 😉

The problem

As I was reading through the nuxt docs about LCP optimisation and tried implementing it, I hit a wall. 🧱

📘 docs link

So here to everyone who has the same issue that I had.

We are using pnpm as our package manager, so the path to the entry.js doesn't look like

node_modules/nuxt/dist/app/entry.js

but more like

node_modules/.pnpm/nuxt@3.12.4_@parcel+watcher@2.4.1_@types+node@18.19.41_eslint@8.57.0_ioredis@5.4.1_magicast@0_sp6oxyoamdsk3zyrb4wztbq7li/node_modules/nuxt/dist/app/entry.js

The solution

To hardcode this path will cause an issue as soon as you update any package that is in that pathname.

To circumvent this you can update the hook to do the following:

import { readdirSync } from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';

export default defineNuxtConfig({
  hooks: {
    'build:manifest': (manifest) => {
      // find the app entry, css list
      const baseModulesFolder = path.dirname(fileURLToPath(import.meta.url)) + '/node_modules/.pnpm/';
      const nuxtFolder = readdirSync(baseModulesFolder).find((file) => typeof file === 'string' && file.startsWith('nuxt@'));
      const entryFilePath = `node_modules/.pnpm/${nuxtFolder}/node_modules/nuxt/dist/app/entry.js`;

      const css = manifest[entryFilePath]?.css;
      if (css) {
        // start from the end of the array and go to the beginning
        for (let i = css.length - 1; i >= 0; i--) {
          // if it starts with 'entry', remove it from the list
          if (css[i].startsWith('entry')) css.splice(i, 1);
        }
      }
    },
  },
})

I hope this helps anyone.

Have a great day and keep on coding 🧑‍💻

Use SvelteKit to create npm packages

Domenik Reitzner — Thu, 10 Feb 2022 11:03:11 +0000

I have been using svelte for about 3 years and have had a lot of fun with it. The simplicity of using it, or the ability to jump into the REPL and trying out a crazy idea are just some of the great things that make it such a pleasure to use.

The next big thing

Almost a year ago things were about to get even better. SvelteKit, the application framework for Svelte apps was announced and entered public beta. SvelteKit is to Svelte what Next is to React or Nuxt is to Vue.

One of the hidden features included with this new tool-belt for building Svelte Apps is the command svelte-kit package.

This command will auto-generate a package from your src/$lib folder (this is a special folder for your shared code/components in SvelteKit) that you can publish directly to npm.

But I am getting ahead of myself. Let'S look at the steps in more detail 😉

Setup your project

To get started run npm init svelte@next my-app.

I would recommend to select the following:

skeleton project
typescript
prettier
eslint

as they should make your live easier especially when working in teams.

There is one more package you need to add by running npm i -D svelte2tsxin your my-app folder.

Now you should be set to start with your first changes.

Good first adaptions

I have developed a mental list of things that I want to do right in the beginning, as I tend to forget them when I jump into the thick of things.

The first thing I take care of is to make some adaptions to the package.json. Specifically I make sure that the package name is correct (had some upsis there). It also makes sense to add a namespace (I created a svackages organisation for all my svelte packages).

Other good things to add are the fields repository, bugs and license.

To finish off this part, I do a rudimentary documentation in the README.md of how my code should work. This step is important as it will be the info shown on npm as soon as you publish. Ideally I will already add REPL url, where my future package will be easy to try out.

Add your code

The next part should be the easiest. Add all the code that you want to publish in src/$lib. To make it easier on yourself, you can use the src/routes folder to test out your package. This is also a nice way of documenting your package and possibly even publishing it as a standalone site.

Publish it

The last few steps to get your library published are actually pretty easy.

run npm run package to generate your package
Make sure that you are logged into npm (npm login)
the last thing to do is to run npm publish ./package (hopefully you have two factor authentication and need to put in your code as well)

Now you have successfully published your (first) package with SvelteKit. 🥳

Happy coding and I hope you have learned something.

Svelte and Localstorage remote

Domenik Reitzner — Mon, 12 Apr 2021 11:53:23 +0000

Idea 💡

localStorage is a very versatile part of web development for persisting data, but it can do so much more.

Every change fires a storage event on the window Object across tabs, which means it can be used as remote. 🤯

Potential uses 🛠

Video controller in a small popup window
Speaker view of presentation slides

Implementation ⚗

I used svelte-kit for the example (see links) 😉
The actual amount of code needed for this pattern is very minimalistic. On the remote side we only need to set a localStorage item and this in turn will trigger the event which we can listen to.

<button
  on:click={() => localStorage.setItem('signal', 'message')}>
  Set 'message'
</button>

The receiver is a little more work, but svelte is our friend and helper 💪

<script>
  let signal = '';
  const listener = () => {
    const value = localStorage.getItem('signal');
    if (!value) return;
    signal = value;
  };
</script>
<svelte:window on:storage={listener} />

Received Signal: {signal}

If you open the remote and the receiver (same browser and base URL) you should be able to send and receive signals across tabs without any network request.

⚠ because of security issues localStorage is blocked on the svelte REPL, so it will not work there

I hope you found this entry interesting and learned something new ☺

Links 🔗

Svelte and Cloudflare Pages

Domenik Reitzner — Tue, 09 Mar 2021 09:30:11 +0000

As I was writing this blog post Cloudflare Pages was still in open beta.

So here is a quick tour on how you can set up your svelte project on cloudflare pages.

Basic setup

First things first.
Login to your cloudflare account (if you don't have one yet, it is really easy to set one up).
Select the icon on the right for cloudflare pages.

After that you can create a new project.

Connect Repository & build

To continue you need to connect your github repo to cloudflare pages. After that you can select one of your (svelte) repositories.

On the next page you can set you project name and build settings. There is already a configuration for svelte that you can select from the dropdown. If needed you can adapt the values to your specific setup.

The first build took a little longer, so no need to worry if it is not immediately done (I guess this is due to the fact that the first setup of your instance takes some time).

Add a custom domain

So after your first build you can access your site on your-project.pages.dev. If you have your domain set up with cloudflare you can easily set up a custom domain for your project.

Just go to your pages dashboard and select your project.
After that you can go to the custom domains tab.

After that your svelte site is ready to shine on cloudflare pages.

Links

Repo used:

dreitzner / svelte-cloudflare-pages

Looking for a shareable component template? Go here --> sveltejs/component-template

svelte app

This is a project template for Svelte apps. It lives at https://github.com/sveltejs/template.

To create a new project based on this template using degit:

npx degit sveltejs/template svelte-app
cd svelte-app

Note that you will need to have Node.js installed.

Get started

Install the dependencies...

cd svelte-app
npm install

...then start Rollup:

npm run dev

Navigate to localhost:5000. You should see your app running. Edit a component file in src, save it, and reload the page to see your changes.

By default, the server will only respond to requests from localhost. To allow connections from other computers, edit the sirv commands in package.json to include the option --host 0.0.0.0.

If you're using Visual Studio Code we recommend installing the official extension Svelte for VS Code. If you are using other editors you may…

View on GitHub

Copy the name of your current branch

Domenik Reitzner — Thu, 18 Feb 2021 09:08:41 +0000

I just wanted to share a quick helper that I have been using for a while.

History

One of the projects I work at has some nicely set up development servers (different variations of cms and business api).

So whenever we need to test a specific configuration on a feature that we are developing, we will put that branch on one of the development servers. For that to happen we need to adapt the configuration inside of the build server and change the branch name.

So after multiple times of changing the branch name, waiting for the build to finish, testing on the server, not seeing my changes and finding out, that I had a typo in the name, I decided I'd hunt for a better solution.

The easy fix I've used ever since.

This is the easy solution I found:

// windows
git branch --show-current | clip

// macOS
git branch --show-current | pbcopy

I hope ths makes your live easier and you can focus more of your time on the things you love to do.

Credits

cover: https://unsplash.com/@yancymin

Svelte and Salesforce Marketing Cloud

Domenik Reitzner — Fri, 18 Dec 2020 10:15:45 +0000

Goal

use svelte for a simple questionnaire for a Salesforce Marketing Cloud Cloud Page
generate JSON Server Side for questionnaire data
submit data with post to Salesforce Marketing Cloud

Use Svelte

To get this up and running it is actually pretty easy.

Start with your standard svelte starter project.

I just defined a custom container as target (#questionaireContainer) and specified the structure of the questionnaire data and it's container (script#questionaireData). The data may vary for your use case, but here is mine:

[{
    "id": "string",
    "title": "string",
    "question": [
        {
            "id": "string",
            "title": "string",
            "position": 1,
            "options": [
                    {
                        "id": "string",
                        "value": "string"
                    }
                ]
        }
    ]
}]

On startup you just get the data from the script tag and parse it.

const data: Idata = JSON.parse(
  document.querySelector('#questionnaireData').innerHTML
)[0];

Now you can work with that data and implement your style. After building your project you need to go to WebStudio > Cloud Pages

and create new code resources inside one of your collections.

Now you can grab that URL and put it as script and style tags into your micro site.

Generate JSON data

We are using a data extension for the questionnaire configuration. We simply made a for loop that goes over each row in our data extension and generates our data in our predefined format.

<script id="questionnaireData" type="application/json">%%[
  set @json=Concat('[{"questions": [')
  if @questionRowCount>0 then
    for @j=1 to @questionRowCount do
      /* magic here*/
    next
  endif
  set @json = Concat(@json,@questionJson, ']}]')
]%%
%%=v(@json)=%%
</script>

Submit data with POST

So this one was a bit tricky to figure out, as the Salesforce documentation is not really forthcoming about how they can handle POST requests in their "beautiful" scripting language (amp script).

So inside our svelte app we just need to do a post with form values (form submit the js way).

const postData = () => {
  const body = new URLSearchParams({
    currentQuestion: question.id,
    currentAnswer: $selectedStar[step] || '',
  });
  const options = {
    method: 'POST',
    body,
  }
  fetch(data.url, options)
    .catch((err) => console.error(err));
};

In the cloud page we just watch for a request parameter (currentQuestion).

if RequestParameter("currentQuestion") !="" then
   set @currentQuestion = RequestParameter("currentQuestion")
   set @currentAnswer =  RequestParameter("currentAnswer")

   set @isPost=true

   if RequestParameter("currentAnswer")!="" then
      /*Save answer in DE*/
   endif
endif

So now we have a way to get data from the marketing cloud into a frontend framework like svelte and also POST data back into it.

Conclusion

It is really nice to use a frontend framework inside of the salesforce marketing cloud, as it is so much better than the alternative, but it is not as easy as I would desire it to be.

Deployment is currently only possible to be done manually (I might have an idea to work around that in the future, but I'm not there yet).

Another drawback is that writing logic in AMP script is just so cumbersome. Everything that goes beyond reading/writing some data from/to a data extension gets complicated really fast.

I hope you enjoyed the read and could take away something.

Svelte and kentico kontent.ai

Domenik Reitzner — Fri, 11 Dec 2020 13:02:44 +0000

This blog post is about adding preview functionality to server-side-rendered CMS content from kentico kontent.ai (in my case we used Salesforce commerce cloud for the rendering). If you use already client rendering for your CMS content, than you don't need this, just add a preview config to your project.

Index

Get your main site ready
Proxy server with polka
Sveltify your site
Make the preview content togglable
Add more CMS items

Get your main site ready

One prerequisite for this whole shenanigan to actually work, is that you have your live site up and running.
Another important step is that you have a way of referencing your ssr content to an kontent.ai id. The way I did it was by using data-system-id in the ssr site.

Proxy server with polka

The node server (I used polka, but express or any similar should work as well) is a very simple one.
I check if I get a call with a ?previewId={id}, which will have the kentico id.

const dir = join(__dirname, '../public'); //dir for public
const serve = serveStatic(dir);

polka()
    .use('/preview', serve)
    .get('*', async (req, res) => {

        let url = req.originalUrl;
        const isMainRequest = url.match(/(\?|&)previewId=/) !== null;
        // some magic 🦄
    })
    .listen(PORT, (err) => {
        if (err) throw err;
        console.log(`> Running on localhost:${PORT}`);
    });

All requests, that are not our main request we will just proxy.

if (!isMainRequest) {
     return request
         .get(url)
         .auth(usr, pwd, false) // if server needs basic auth
         .pipe(res);
}

For our main request it is important, that we remove our custom Url parameter

const toRemove = url.match(/[\?|&](previewId=.*?$|&)/)[1];
url = url
    .replace(toRemove, '')
    .replace(/\/\?$/, '');

After that we can handle our main request and inject our js/css bundles at the end of our html

// get requested site from live server
const resp = await fetch(url, {headers});
let text = await resp.text();

// add script tag before </body>
if (text.includes('<html')) {
    const bundles = `
        <script src="/preview/bundle.js" async></script>
        <link rel="stylesheet" href="/preview/bundle.css">
    `;
    if(text.includes('</body>')) {
        text = text.replace('</body>', `${bundles}</body>`)
    } else {
        // cloudflare eg. minifies html
        // by truncating last closing tags
        text += bundles;
    }
}
// return response
return res.end(text);

Sveltify your site

The best choice for the frontend in my opinion (especially for such small an powerful tool) is svelte.

I leaves a small footprint comes with huge capabilities and is ideal if you want to run a tool on top of another site.

The basic svelte setup (with ts) looks something like this:

<!-- App.svelte -->
<script lang="ts">
    import { onMount } from 'svelte';

    // INIT VARS
    let preview = true;
    let addMode = false;
    let toggleFuncs = new Map();
    let arrayOfCmsNodes = [];
    let overlays = [];

    onMount(() => {
        // some init stuff
    });
</script>

<main>

</main>

CSS can be totally custom. In my project I put the tools in the bottom right corner, but this is just my preference, so I'll leave them out.

In the onMount function I initialize the app by getting the previewId and setting up all available dom nodes that have cms capability. (in my case I excluded child cms components)

// App.svelte
onMount(() => {
    // get param from url
    const url = new URL(document.URL);
    const id = url.searchParams.get('previewId');
    loadPreview(id);
    const tempArr = [];
    document.querySelectorAll('[data-system-id]')
        .forEach((node: HTMLElement)  => {
            if (node.dataset.systemId === id) return;
            // for nested this needs to exclude children data-system-id
            if((node.parentNode as HTMLElement).closest('[data-system-id]') !== null) return;
            tempArr.push(node);
        });
    arrayOfCmsNodes = tempArr;
});

As you can see, the next step was to call loadPreview(id). This will get the preview data from Kontent.ai

// App.svelte
import { getPreviewContent } from './service/kontent';
import { getToggle } from './service/toggleFunctionGenerator';
const loadPreview = async (id: string) => {
    if (!id) return;
    const content = await getPreviewContent(id);
    if (!content?.items?.length) return;
    const toggle = getToggle(id, content);
    if (!toggle) return;
    toggleFuncs.set(id, toggle);
    if(preview) toggle();
}

To get the content you just need to fetch the content by id from https://preview-deliver.kontent.ai/${projectId}/items?system.id=${key} by setting an authorization header with your preview key.

const headers = {
    'authorization': `Bearer ${previewKey}`
};

Make the preview content togglable

As we want the content to not be only replaced, but toggle between live and preview version, we need to generate a toggle function.

For switching between those states I created a simple toggle switch and function.

<!-- App.svelte -->
<script lang="ts">

    import Toggle from './components/Toggle.svelte';

    const togglePreviews = () => {
        preview = !preview
        toggleFuncs.forEach(func => func());
    }

</script>

<main>
    <Toggle
        {preview}
        {togglePreviews} />
</main>

Setting up the toggle function was a little bit more complex, but in the end it is really easy to add more entries.

// .service/toggleFunctionGenerator.ts
import {
    replaceText,
} from './replaceContent';

import {
    getToogleDataByType,
} from './toggleConfig';

const getNodeBySystemId = (id: string) => document.querySelector(`[data-system-id='${id}']`);

const handleType = (type: string, id: string, elements: IKElements, modularContent: IKModularContent): { (): void} => {
    const node = getNodeBySystemId(id);
    if (!node) return null;

    const {
        textReplace,
    } = getToogleDataByType(type, elements);

    const children = Object.keys(modularContent).length
        ? Object.entries(modularContent)
            .map(([key, value]) => handleType(value.system.type, value.system.id, value.elements, {}))
            .filter((child) => !!child)
        : [];

    const toggleFunc = () => {
        if (textReplace) replaceText(node, textReplace);
    };

    return toggleFunc;
};

export const getToggle = (id: string, content: IKContent) => {
    const item = content.items[0];
    return handleType(item.system.type, id, item.elements, content.modular_content)
};

By wrapping everything into a toggle function, we keep the state available inside of it. As kontent.ai will return a lot of data that will not be used, I decided to explicitly save the data that I need. I this inside of getToogleDataByType.

// .service/toggleConfig.ts

// in my project I have 6 different data generators, so they ended up in a new file
const getGenericElements = (elements: IKElements, keyMapper: IKeyValue): IReplacer[] => {
    const tempArr: IReplacer[] = [];
    Object.entries(keyMapper).forEach(([key, querySelector]) => {
        const data = elements[key]
        if (!data) return;
        tempArr.push({
            querySelector,
            value: data.value,
        });
    });
    return tempArr;
};

// Toggle Data Config
const myType = (elements: IKElements): IToggleData => {
    const textKeyMapper: IKeyValue = {
        my_title: '.js-my_title',
    };

    return {
        textReplace: getGenericElements(elements, textKeyMapper),
    }
};

export const getToogleDataByType = (type: string, elements: IKElements): IToggleData => {
    const callRegistry = {
        myType: myType,
    }

    const caller = callRegistry[type];
    return caller
        ? Object.assign({}, caller(elements))
        : {};
}

Each replacer will give us an array with objects that will match the preview value with the dom selector (or whatever other stuff you can think of).

So how does the data generation actually translate to updating the dom when the toggle function is called?
It is basically just getting and saving the old value and setting the new one.

// .service/replaceContent.ts
const getElementByQuerySelector = (node: Element, querySelector: string): any => querySelector === null
    ? node
    : node.querySelector(querySelector);

export const replaceText = (node: Element, textElements: IReplacer[]) => {
    textElements.forEach(({querySelector, value}, i) => {
        const element = getElementByQuerySelector(node, querySelector);
        if (!element) return;
        const old = element.textContent;
        element.textContent = value;
        textElements[i].value = old;
    });
};

So we've got the basics up and running. But to only have one id previewed is a little boring.

Add more CMS items

As we alread have an array of cms nodes, setting this up should be fairly easy. ☺
We just need an overlay and handle the add click with the already existing setup.

<!-- App.svelte -->
<script lang="ts">
    import AddButton from './components/AddButton.svelte';
    import AddBox from './components/AddBox.svelte';

    const handleAddClick = (idToAdd: string) => {
        handleAddMode();
        loadPreview(idToAdd);
        arrayOfCmsNodes = arrayOfCmsNodes.filter((node: HTMLElement) => node.dataset.systemId !== idToAdd);
    }

    const handleAddMode = () => {
        addMode = !addMode;
        if (addMode) {
            arrayOfCmsNodes.forEach((node: HTMLElement) => {
                const {top, height, left, width} = node.getBoundingClientRect();
                overlays.push({
                    id: node.dataset.systemId,
                    top: top + window.scrollY,
                    height: height,
                    left: left,
                    width: width,
                });
            })
            overlays = overlays;
        } else {
            overlays = [];
        }
    }
</script>

<main>
    {#if arrayOfCmsNodes.length}
        <AddButton
            {addMode}
            {handleAddMode} />
    {/if}
</main>
{#each overlays as {id, top, height, left, width}}
    <AddBox 
        {id}
        {top}
        {height}
        {left}
        {width}
        {handleAddClick} />
{/each}

I know this part was by far the easiest one, but is adds a lot of value to the functionality, so I wanted to include it here.

Thank you for reading and I hope you can take away something or are inspired for your own project.

Credits

cover image: https://unsplash.com/@marvelous

Svelte and TS + comlink

Domenik Reitzner — Mon, 10 Aug 2020 15:20:45 +0000

The project

I have been trying stuff with svelte and doing some projects with it for a little over a year now. I have liked it from the start and still do and enjoy the simplicity and speed of it.

3 weeks ago I had the idea to do another little side project and build a guitar tuner with svelte (a svelte tuner if you'd like). I wanted it to look similar to my physical tuner that I use when I play live with my guitar.

Typescript

It was right around the time svelte started supporting typescript out of the box. So I thought to myself: "That is a nice project to start learning typescript". And I must tell you the transition was really easy (and even saved me from some bugs along the way).

Some problems that I encountered

share my interfaces among files
import svgs

Share interfaces

As a total beginner in ts I started out with exporting my interfaces instead of having them in my @types folder. I thought it might be worth mentioning for all those who start out with typescript 😉

import svgs

another thing that I encountered (in a different project) was when I tried importing svgs it gave me a ts error. The solution that I found, is to have an index.d.ts inside ./@types with the following:

declare module "*.svg" {
    const content: any;
    export default content;
}

Using comlink

I have heard about comlink in some js youtube tutorial and wanted to try it out for the longest time. With raw audio processing this seemed to be the perfect fit to run the task off the main thread.
It was really easy to integrate (also with typescript).

MyComponent.svelte => script

import * as Comlink from "comlink";

const worker = new Worker("/myWorker.js");
const instance: Comlink.Remote<IWorkerConstructor> = Comlink.wrap(
  worker
);
const localInstance = await new instance();

...

const result: number = await localInstance.calculateNumber()

myWorker.js

import * as Comlink from "comlink";

class Worker implements IWorker{
    calculateNumber() {
        return Math.random();
    } 
}

Comlink.expose(Worker);

The syntax is really easy and provides a powerful enhancement to apps that run expensive calculations.

Conclusion

Integrating workers with comlink into apps, that do some calculations keeps the main thread healthy especially on low end devices (which more and more people are using these days).
Also using ts with svelte seems a really powerful combination, without loosing its simplicity. I guess this will further cement svelte's position in it's approach to an awesome development experience.

So, try stuff out, have fun and enjoy web deving 🎉

Links:

My music:

Make unused public methods private in services

Domenik Reitzner — Fri, 07 Aug 2020 12:04:49 +0000

We have finally reached the end of the line (maybe 😋).

Building upon the shoulders of our past code

We have already invested a lot of time and afford into streamlining our code bundle.

In this final puzzle piece we will look at all our services and see if we find any methods and members that we only use privately.

Gather all the used services methods

This is pretty similar to searching for used methods in templates from the last article. Instead of looking at hmlt files we will search in all of our js files.

Also the regex (at least in my case) changed a bit.
I now search for all strings that start with a big letter followed with at least one small letter. The whole thing needs to be prefixed with " " (inside none class syntax files), "_" (normal class service injection), or "!" (negated none class syntax).

const matches = string.match(/[ |_|!][A-Z]+[a-z]+[a-zA-Z]*\.[a-zA-Z0-9_-]+/gm);

Replacing that and adapting it to use a different Set as before, we end up with:

const stringsFromJs = new Set();
const tGatherFromJs = () => {
    return gulp.src(packagePath + '/**/*.js', {base: './'})
        .pipe(through.obj((file, _, cb) => {
            const string = file.contents.toString();
            const matches = string.match(/[ |_|!][A-Z]+[a-z]+[a-zA-Z]*\.[a-zA-Z0-9_-]+/gm);
            if (matches) matches.forEach((match) => stringsFromJs.add(match.substring(1)));
            cb();
        }));
};

Oh wait, we can reuse or existing function

As we already loop over all our js files (tFindUnusedPublic) we can just add our service check.

const tFindUnusedPublic = () => {
    const stringToSearchT = [...stringsFromTemplate].join(' ');
    const stringToSearchJs = [...stringsFromJs].join(' ');

    // get public from js files
    return gulp
        .src(packagePath + '/**/*.js', {base: './'})
        .pipe(through.obj((file, _, cb) => {
           // ... some stuff we already do
                // components and directices
                let baseName = string.match(/controllerAs: '(.*)'/);
                if (baseName) { /*...*/ }

                // services
                let serviceName = string.match(/\.service\('(.*)'/);
                if (serviceName) {
                    serviceName = serviceName[1];
                    const uniquePublic = getPublic(string);
                    string = searchInString(uniquePublic, serviceName, stringToSearchJs, string, false);
                }
            // ... more some stuff we already do
        }))
        .pipe(gulp.dest('./'));
};

So with this little addition we now also check our services for unused methods and members.

Ideally our compiled code base should be smaller now (I saved about 100kB) and our users should be a bit more happy.

I hope that you where also able to squeeze a little more performance out of your angularjs project.

All the best and enjoy frontending 😉

happy coding🎉

Make unused public methods private in components and directives

Domenik Reitzner — Fri, 17 Jul 2020 08:42:01 +0000

This is part six of seven.

The great clean up

We have looked at many different ways to improve code and build quality of our code base. One of the greatest improvements (at least in my case) was to minify private methods. Today we are going to milk this cow even further and look for unused public methods/members in our code base.

Where are you?

We should be able to find all public methods and members that we use inside of our components and directives in our views/templates. So the first step in finding all unused methods is to find those we actually do use.

A great tool at our disposal for this is regex.
In my case all controller names start with a lower case letter followed by any type of letter and a .

/[a-z]+[a-zA-Z]+\./gm

All my methods/members (after the .) follow almost the same pattern. They don't have to start with a lower case letter and and can also have _ and -. So we end up with:

/[a-z]+[a-zA-Z]+\.[a-zA-Z_-]+/gm

I wanted to reduce my list even further and thought about all the allowed symbols that are allowed to be before my expression. So my final glorious reges looks like this

/([ |:|'|"|\{|!|\[])([a-z]+[a-zA-Z]+\.[a-zA-Z_-]+)/gm

Stitching it all together:

// new set to get unique matches across all templates
const stringsFromTemplate = new Set();
const tGatherFromTemplates = () => {
    // all  your templates
    const src = [
        packagePath + '/**/*.html',
    ];
    return gulp.src(src, {base: './'})
        .pipe(through.obj((file, _, cb) => {
            // look through content of each file
            const string = file.contents.toString();
            // find all matches
            const matches = string.match(/([ |:|'|"|\{|!|\[])([a-z]+[a-zA-Z]+\.[a-zA-Z_-]+)/gm);
            // add matches to our set
            if (matches) matches.forEach((match) => stringsFromTemplate.add(match.substring(1)));
            cb();
        }));
};

Now we should have a list of all used combinations of controller.method (if you are using $ctrl in your project you need to adapt your regex to take this into account)

I'm gonna fix you up

⚠ this step will edit your files, so be careful

As we might encounter some methods/members, that have to be replaced manually (bad naming convention in files, etc.), I started with setting up some Sets (again 😉) and make a regex searchable string out of my unique strings from Template.

const unusedPublic = new Set();
const unusedManual = new Set();
const tFindUnusedPublic = () => {
    const stringToSearchT = [...stringsFromTemplate].join(' ');

    // get public from js files
    return gulp
        .src(packagePath + '/**/*.js', {base: './'})
        .pipe(through.obj((file, _, cb) => {
            const newFile = file.clone();
            let string = newFile.contents.toString();

            // magic about to happen...

            // overwrite contents of vinyl with new buffer
            newFile.contents = Buffer.from(string);
            cb(null, newFile);
        }))
        .pipe(gulp.dest('./'));
};

The approach that I took was to check if the file is a class and then extract the controller name.

const isClass = string.match(/class [A-Z]/);
if (isClass) {
    // components and directices
    let baseName = string.match(/controllerAs: '(.*)'/);
    if (baseName) {
        // get name from capture group
        baseName = baseName[1];
        // TODOs:
        // - get public methods/members
        // - replace unused
    }
}

So the next step is to extract all public methods/members. I extracted this into it's own function, as I'll reuse it for the services in the upcoming article.

const getPublic = (string) => {
    // identation is your friend
    // method starts with lower case letter
    let matchesPublicMethods = string.match(/^ {4}[a-z][a-z0-9_]+\(/gim) || [];
    matchesPublicMethods = matchesPublicMethods
        // remove indent
        .map((m) => m.slice(4, -1))
        // filter out constructor and edge cases
        .filter((m) => !(m.match('constructor') || string.match(new RegExp(`function ${m}`, 'gim'))));
    // get all members and remove "this."
    let matchesPublicMembers = string.match(/this\.[a-z][a-z0-9_]+/gim) || [];
    matchesPublicMembers = matchesPublicMembers.map((m) => m.slice(5));
    // combine and sort descending by length to circumvent bad naming
    const combinedPublic = [...matchesPublicMethods, ...matchesPublicMembers]
        .sort((a, b) => b.length - a.length);
    return new Set(combinedPublic);
};

All that is left to do is to search and replace unused methods/members and stitch it all together.

const searchInString = (uniquePublic, baseName, inString, file, replace) => {
    uniquePublic.forEach((m) => {
        const search = `${baseName}.${m}`;
        // check if method/member is in generated list
        if (!inString.match(search)) {
            // add to set
            unusedPublic.add(search);
            // handle pre and postfixes for manual replacement
            if (file.match(new RegExp(`[_a-zA-Z']+${m}`, 'gm'))
                || file.match(new RegExp(`${m}[a-zA-Z0-9']+`, 'gm'))) unusedManual.add(search);
            // if replace is enabled replace it
            else if (replace) file = file.replace(new RegExp(m, 'gm'), `_${m}`);
        };
    });
    // return modified file
    return file;
};

if we put it all together we come up with the following:

const stringsFromTemplate = new Set();
const tGatherFromTemplates = () => {
    const src = [
        packagePath + '/**/*.html',
    ];
    return gulp.src(src, {base: './'})
        .pipe(through.obj((file, _, cb) => {
            const string = file.contents.toString();
            const matches = string.match(/([ |:|'|"|\{|!|\[])([a-z]+[a-zA-Z]+\.[a-zA-Z_-]+)/gm);
            if (matches) matches.forEach((match) => stringsFromTemplate.add(match.substring(1)));
            cb();
        }));
};

const getPublic = (string) => {
    let matchesPublicMethods = string.match(/^ {4}[a-z][a-z0-9_]+\(/gim) || [];
    matchesPublicMethods = matchesPublicMethods
        .map((m) => m.slice(4, -1))
        .filter((m) => !(m.match('constructor') || string.match(new RegExp(`function ${m}`, 'gim'))));
    let matchesPublicMembers = string.match(/this\.[a-z][a-z0-9_]+/gim) || [];
    matchesPublicMembers = matchesPublicMembers.map((m) => m.slice(5));
    const combinedPublic = [...matchesPublicMethods, ...matchesPublicMembers]
        .sort((a, b) => b.length - a.length);
    return new Set(combinedPublic);
};

const searchInString = (uniquePublic, baseName, inString, file, replace) => {
    uniquePublic.forEach((m) => {
        const search = `${baseName}.${m}`;
        if (!inString.match(search)) {
            unusedPublic.add(search);
            // handle pre and postfixes
            if (file.match(new RegExp(`[_a-zA-Z']+${m}`, 'gm'))
                || file.match(new RegExp(`${m}[a-zA-Z0-9']+`, 'gm'))) unusedManual.add(search);
            else if (replace) file = file.replace(new RegExp(m, 'gm'), `_${m}`);
        };
    });
    return file;
};

// find public services that should be private
const unusedPublic = new Set();
const unusedManual = new Set();
const tFindUnusedPublic = () => {
    const stringToSearchT = [...stringsFromTemplate].join(' ');

    // get public from js files
    return gulp
        .src(packagePath + '/**/*.js', {base: './'})
        .pipe(through.obj((file, _, cb) => {
            const newFile = file.clone();
            let string = newFile.contents.toString();

            const isClass = string.match(/class [A-Z]/);
            if (isClass) {
                // components and directices
                let baseName = string.match(/controllerAs: '(.*)'/);
                if (baseName) {
                    baseName = baseName[1];
                    const uniquePublic = getPublic(string);
                    // set to true if you are certain you want to replace
                    string = searchInString(uniquePublic, baseName, stringToSearchT, string, false);
                }
            }

            // overwrite contents of vinyl with new buffer
            newFile.contents = Buffer.from(string);
            cb(null, newFile);
        }))
        .pipe(gulp.dest('./'));
};

const tDebug = (cb) => {
    console.log(unusedPublic.size, unusedPublic);
    console.log(unusedManual.size, unusedManual);
    cb();
};

exports.findUnusedPublic = series(
    tGatherFromTemplates,
    tFindUnusedPublic,
    tDebug,
);

Feel free to comment down below if you can think of any edge cases that are not included (one I already know about is getters and setters).

Coming up next

In the next and final part of this series we will take a look at applying the same procedure to the services, by using some of the functions we implemented today.

Make service injection private

Domenik Reitzner — Tue, 14 Jul 2020 15:00:41 +0000

This is part five of seven.

Prerequisits

This part is very specific to how this project is set up and depends on what has been done in step four.

Our components/directives/services use the class syntax and bind the injected service to the class to keep the scope clean from services.

class AwesomeComponent {
    constructor(MyService) {
        Object.assign(this, {MyService});
    }
}

The bundle served

After the build process our code will look something like this:

var AwesomeComponent = function() {
    AwesomeComponent.$inject = ['MyService'];

    function AwesomeComponent(MyService) {
        _classCallCheck(this, AwesomeComponent);
        Object.assign(this, {
            MyService: MyService
        });
    }
}

Some of you might already see the potential her 😉

The way to privatization

In the last part we have looked at the possibility to minify our private methods (in my project I declare them with _ in the beginning). As our services are exclusively used within the component/directive/service, we could argue that they are injected private Methods.

So the way to optimize our class would be:

class AwesomeComponent {
    constructor(MyService) {
        Object.assign(this, {
            _MyService: MyService,
        });
    }
}

It would be really boring and prone to mistakes to do all those changes by hand.

So here is my attempt to rework the services with a gulp task (once).
It follows those steps:

get all js files
look for the object assign pattern (objectAssignRegex)
if we have no changes we continue without changes
get all the services from the capture group and split them into an array
map through the services and do the private method trick
join the array with line breaks (\n)
wrap them in formatted object assign wrapper
Replace the old Object.assign contents with the generated string
replace all this.Service with this._Service (careful of $)
overwrite the contents

// make services private
const tPrivateService = () => {
    const objectAssignRegex = /Object.assign\(this, \{(.+?)\}/m;
    return gulp
        .src(packagePath + '/**/*.js', {base: './'})
        .pipe(through.obj((file, _, cb) => {
            const newFile = file.clone();
            let string = newFile.contents.toString();

            const match = string.match(objectAssignRegex);
            if (match) {
                console.log('Working on ', file.basename);

                const services = match[1]
                    .split(', ')
                    .map((service) => service.trim());

                const newServices = services
                    .map((service) => `            _${service}: ${service},`).join('\n');

                const newObjectAssign = match[0]
                    .replace(match[1], newServices)
                    .replace('{', '{\n')
                    .replace('}', '\n        }');

                string = string.replace(objectAssignRegex, newObjectAssign);

                services.forEach((service) => {
                    string = string.replace(new RegExp(`this.${service.replace(/\$/g, '\\$')}`, 'gm'), `this._${service}`);
                });

                // overwrite contents of vinyl with new buffer
                newFile.contents = Buffer.from(string);
            }
            cb(null, newFile);
        }))
        .pipe(gulp.dest('./'));
};

exports.privateService = series(tPrivateService);

In my case, that worked for almost all files. In some cases I had the Object.assign implemented in a none standard way (with additional var declarations or line breaks).
Before you submit check all your changes.

Enjoy your privatized service injection 🎉

Coming up next

Privatize unused public methods in components/directives

Uglify private methods/members

Domenik Reitzner — Fri, 10 Jul 2020 08:38:50 +0000

This is part four of seven.

Why angularjs, why....?

One problem with angularjs is that it transforms methods/functions inside of components/directives/services into an object. The result from that is that minifiers can't replace private methods automatically (we use _privateMethod in our project as convention).

_makeAwesome() {
    this.isAwesome = true;
}

will become (after angularjs-annotate)

{
    key: '_makeAwesome',
    value: function _makeAwesome() {
        this.isAwesome = true;
    }
}

Because _makeAwesome is now also represented as string we can't use a normal uglify (even with regex options).

Fix it before annotation

One day I had an epiphany, why not switch them out before the annotation happens? After searching for some npm package and finding nothing, I decided to just write my own gulp replace function. For each file I needed to replace all private members and methods with a unique id (for that file).

gulp
    .src(src)
    .pipe(p.sourcemaps.init())
    // replace private function names with stringId
    .pipe(through.obj((file, _, cb) => {
        const newFile = file.clone();
        let code = newFile.contents.toString();
        const ids = new StringIdGenerator();
        // gets all private function declarations/methods in a file
        let matchesPrivateFunctions = code.match(/( _[a-z0-9\$_]+\()/gim);
        if (matchesPrivateFunctions) {
            // remove bracket and space from match to not miss this._func.bind(this)
            matchesPrivateFunctions = matchesPrivateFunctions
                .map((match) => match.replace(/\(|\s/g, ''))
                .sort((a, b) => b.length - a.length);
            // replace all the matchesPrivateFunctions
            matchesPrivateFunctions.forEach((match) => {
                code = code.replace(new RegExp(match, 'gm'), ids.next());
            });
        }
        // get all private Members inside of classes
        const matchesPrivateMembers = code.match(/this._[a-z0-9\$_]+/gim) || [];
        // sort with longest names on top to not miss this._privates after replacing this._priv 
        const uniquePrivateMembers = new Set(matchesPrivateMembers.sort((a, b) => b.length - a.length));
        if (uniquePrivateMembers.size) {
            uniquePrivateMembers.forEach((match) => {
                // replace without this. prefix => bindings might have members with `_`
                match = match
                    .replace('this.', '')
                    .replace('$', '\\$');
                code = code.replace(new RegExp(match, 'gm'), ids.next());
            });
        }

        // overwrite contents of vinyl with new buffer
        newFile.contents = Buffer.from(code);
        cb(null, newFile);
    }))
    .pipe(p.babel({
        // babel and angularjs-annotate
    }))
    // ... some more build stuff

String id generator

here is the id generator that I used:

class StringIdGenerator {
    constructor(chars = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ') {
        this._chars = chars;
        this._nextId = [0];
    }

    next() {
        const r = [];
        for (const char of this._nextId) {
            r.unshift(this._chars[char]);
        }
        this._increment();
        return r.join('');
    }

    _increment() {
        for (let i = 0; i < this._nextId.length; i++) {
            const val = ++this._nextId[i];
            if (val >= this._chars.length) {
                this._nextId[i] = 0;
            } else {
                return;
            }
        }
        this._nextId.push(0);
    }

    * [Symbol.iterator]() {
        while (true) {
            yield this.next();
        }
    }
}

Bonus tip

In development we don't really care about this transformation (or any minification). In our project we set a flag (isDev) when it is a local build. So it was relatively easy to wrap this steps in a little helper function.

return gulp
    .pipe(_stepOverOnDevBuild(/*...*/))
    // ...

const _stepOverOnDevBuild = (func) => {
    return isDev
        ? p.util.noop()
        : func;
};

Word of caution:
test with minification/transformation in place before you deploy

Coming up next

Make service injections private (to minify with what we build today)