DEV Community: Mati Andreas

Caching and local storage in PWA

Mati Andreas — Sun, 28 Mar 2021 14:09:12 +0000

Image from https://blog.vuestorefront.io/caching-on-production.

Base technologies like Vue.js, Vuex, and SSR, Service Workers are used in or team to build modern e-commerce frontends. The Vue Storefront framework PWA will be used as the application skeleton for starting a new project.

Whenever it's possible, the service worker cache is in use. For easier service worker cache management, Google provided Workbox is in use.
Service worker cache is used mainly for catalog (category, product, and attribute list and detail) and CMS (pages, blocks, banners).

The Service Workers cache works like we're consistently executing the backend requests to invalidate the local cache. It's the NetworkFirst mode. That means there is virtually no risk of stale / non-invalidated data served from the local cache

https://blog.vuestorefront.io/caching-on-production

LocalStorage or IndexedDB (depending on browser support, abstracted with LocalForage library) are mainly used for storing user-based data like cart, login token, made orders, personal data, other choices (like favorite products).
The user-based cache is updated on demand (on login, viewing order list) and has no end-of-life by default. End-of-life is added case-by-case depending on the business needs.

Step by step estimation guide

Mati Andreas — Sun, 28 Mar 2021 13:49:25 +0000

Why this article

At some point, as a developer, each one of us has had to estimate something. Sometimes estimation is easy, sometimes hard. Easy when you know and hard when there is a lot of unknowns. And it does not matter if you are already using PERT or voting for #NoEstimates.

With this article, I want to give a simple step-by-step guide to those who are just starting or struggling with it.

But note! This document bases on subjective personal experiences. Figures are exemplary and depend highly on a "bunch of work" to be estimated.

Preliminary estimation

A project manager, team lead, architect, etc., walk to you (or slacks, emails, zooms) and ask for an estimation.

1) Take a deep breath, move your mind off from current work and make a preliminary estimation

an hour,
a day,
a week,
a month,
much more.

2) Make yourself and the person who requests the estimation aware of the preliminary estimation and therefore the inaccuracy of the assessment:

b) magnitude 1 hour:

30 seconds spent - preliminary estimation,
5 minutes spent - estimation;

c) magnitude 100 hours:

5 minutes spent - black magic,
1 hour spent - preliminary estimation,
10 hours spent - estimation.

3) Consult with the estimation questioner about the need to spend more time for preliminary estimation or/and estimation.

4) Make a list of key points based on that black magic / preliminary estimation - a technical specification skeleton.

In my opinion, estimation makes us faster developers by forcing us to compose functionality beforehand in our minds - technical specification - thus spearing time in the future, otherwise spending of false keystrokes.

Estimation

The estimation questioner has confirmed preliminary extra time to be spent on getting a more detailed estimation.

Finish the technical specification:

magnitude 1 hour - 2-3 sentences;
magnitude 100 hours - at least two documents:
- to client - adjusted functional requirements,
- to developer - technical specification.

Adjusted functional requirements are a mirrored-back and specified explanation to a customer about this "bunch of work" outcome.
Estimation must also include testing, documentation, and communication!

Work

Follow the technical specification given in the estimation stage.
If you can't follow it or it does not seem complete, notify the estimation questioner ASAP, and then you can decide together either to move back to the estimation stage or not. Similarly, when bugs are reported back by any party, weigh if it is a missing initially planned feature or new feature request.

Improving quality with Git CLI and PhpStorm

Mati Andreas — Fri, 08 Jan 2021 17:14:49 +0000

Bugs are an inevitable part of software and mistakes are an inevitable part of being a human. That can't be denied but should be dealt with. In this post I want to show how I have learned to make lesser mistakes when making code changes and committing them to Git.

Always check your commit
- Use the command line
- Or use PhpStorm
Use UI tool for merging
Don't be over confident

🔍 Always check your commit

You have made some changes and you are ready to commit them. But maybe you were distracted by some event during development or you don't feel your best today. Check and double check what you are committing!

🔍 Use the command line

git status and git diff --staged are the best commands for that.
git status shows you what files changes are ready to commit. In the example there are

two changes staged - ready to commit and
two changes not staged:

you@computer:path/to/your/project$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        deleted:    assets/js/configurator/MessageFactory.js
        modified:   assets/js/tests/configurator/base_logic.test.js

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   assets/js/tests/configurator/base_logic.test.js

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        assets/js/tests/file_is_added.js

☝ Note that base_logic.test.js is in both states!

git diff --staged shows changes to be committed:

you@computer:path/to/your/project$ git diff --staged
diff --git a/assets/js/configurator/MessageFactory.js b/assets/js/configurator/MessageFactory.js
deleted file mode 100644
index e63cfde..0000000
--- a/assets/js/configurator/MessageFactory.js
+++ /dev/null
@@ -1,17 +0,0 @@
-"use strict";
-
-class MessageFactory extends Module {
- truncated for example
-}
-module.exports = MessageFactory;
\ No newline at end of file
diff --git a/assets/js/tests/configurator/base_logic.test.js b/assets/js/tests/configurator/base_logic.test.js
index 940372a..cb9256d 100644
--- a/assets/js/tests/configurator/base_logic.test.js
+++ b/assets/js/tests/configurator/base_logic.test.js
@@ -6,7 +6,7 @@ chai.should();
 chai.use(sinonChai);

 describe('Configurator', function() {
-    describe('Base Logic', function() {
+    describe('Row is changed', function() {
         let MainLoader = require('./main_loader.js');
         let errorMessage = 'Not enough parents or too many child products selected!';
         describe('simple', function() {

☝ Note that there is a easy way to skip some of the changes when staging.

🔍 Or use PhpStorm

Sometimes there are too many or complex changes that git diff is not enough. Most IDE's have views for helping to understand what you have changed.
☝ You should always make commits as small as possible.

To see only added changed files in the project tree, select "Default Changelist" from the project menu:

After selecting "Default Changelist", IDE displays a list of changed and added files, staged for commit:

☝ PhpStorm shows here only changed and new files that are added to git for committing (staged) and not deleted files.

For seeing all files (changed, added, deleted), use Version Control Window:

Right click on a file to see more actions including "Show Diff".

In opened file window, IDE shows you changes made to file:

line added on row 2 ,
line removed between 4-5,
and changed on row 9. When hovering the mouse on a change hinting mark (green rectangle, red triangle, blue rectangle), IDE shows the previous version.

☝ Be aware that IDE does not differ changes that are and are not staged for commit!

You can jump to the next change in a longer file using the keyboard:

You can also compare the full file with "Compare with Same Repository Version" action:

⚡ Use UI tool for merging

Sometimes merge conflicts happen, either when you are squash merging to master, rebasing master to your feature branch or popping changes from stash:

you@computer:path/to/your/project$ git merge --squash origin/KEY-123
Auto-merging templates/configurator/product_configurator.twig
Auto-merging public/build/configurator.product_configurator.js
CONFLICT (content): Merge conflict in public/build/configurator.product_configurator.js
Auto-merging assets/js/tests/configurator/main_loader.js
Auto-merging assets/js/configurator/Data.js
CONFLICT (content): Merge conflict in assets/js/configurator/Data.js
Squash commit -- not updating HEAD
Automatic merge failed; fix conflicts and then commit the result.

If you are not familiarized yourself with vimdiff, then your tool for solving merge conflicts should be IDE.
Use the resolve conflicts action and continue from there and don't be afraid - remember you can always start over if something goes wrong - that's the beauty of Git.

☝ When there is a possibility that stash pop produces merge conflicts, use stash apply instead!
☝ Resolve Conflicts action is only available if any merge conflicts are waiting to be resolved.

💣 Don't be overconfident

During the time between starting and finishing this article I did not bother to check the diff from git --squash merge from really small change and the result after commit and push build error occurred:

Postscript

When you have come so far, you will reach even further. Keep going!

Vuex explained through PHP

Mati Andreas — Wed, 06 Jan 2021 17:35:59 +0000

Vue.js was not around when I and many other developers started to code years ago. Even jQuery was just released. And if you have been using jQuery in front-end as a (PHP backend) developer, understanding all the aspects of Vuex flow could be tricky.

Let's see first this example from real 2007 code, still in production:

$myrow = mysql_fetch_array(mysql_query(
    "select * from albums where number LIKE '"
    . mysql_real_escape_string($_GET['number']) . "'"
));
echo '<b>' . stripslashes($myrow["number"])
    . ' album &#39;' . stripslashes($myrow["name"]) . '&#39;</b>';

In simple PHP page, data is taken from database and printed out.

Now let's try to write it in Vuex + Vue component like flow in PHP:

# VUEX
$GLOBALS['state']['album_data'] = null;

function actionLoadCurrentAlbum(string $number): void
{
    ## Backend API call
    $myrow = mysql_fetch_array(mysql_query(
        "select * from albums where number LIKE '"
        . mysql_real_escape_string($number) . "'"
    ));
    mutationCurrentAlbum($myrow);
}

function mutationCurrentAlbum(?array $albumData): void
{
    $GLOBALS['state']['album_data'] = $albumData;
}

function getCurrentAlbum(): ?array
{
    return $GLOBALS['state']['album_data'];
}

# Vue Component
## Script
actionLoadCurrentAlbum($_GET['number']);
$albumData = getCurrentAlbum();
## template
echo '<b>' . stripslashes($albumData['name']) . ' album &#39;' 
    . stripslashes($albumData['number']) . '&#39;</b>';

And in JavaScript as Vuex and Vue component

new Vuex.Store({
    state: {
        album_data: null
    },
    actions: {
        async loadCurrentAlbum(context, number) {
            const url = `https://example.com/some/path/${number}`
            let response = await (await fetch(url)).json()
            context.commit('setCurrentAlbum', response)
        }
    },
    mutations: {
        setCurrentAlbum(state, albumData) {
            state.album_data = albumData
        }
    },
    getters: {
        getCurrentAlbum: state => {
            return state.album_data
        }
    }
})

Vue.component('album', {
    beforeCreate: function() {
        this.$state.dispatch('loadCurrentAlbum',
            this.$route.params.number)
    },
    computed: mapState([
        'album_data'
    ]),
    template:
        '<b>{{ album_data.name }} album '
        + '&#39;{{ album_data.number }}&#39;</b>'
})

Now we have example translated from year 2007 PHP page.

A memory leak in Doctrine

Mati Andreas — Mon, 28 Dec 2020 11:28:35 +0000

When building a new PHP application with Doctrine (doctrine/orm 2.7.4), our team discovered that in some long-running commands memory usage was building up.
We went through the basic checklist:

Application is running in production mode - our experience has shown, when is not, then Doctrine SQL logger could eat up memory - checked.
Doctrine clear is called regularly - when it's not, then objects persist in Doctrine internal in-memory cache - checked.
No other code in our application was not obviously building its own memory cache - checked.

Due to the amount of data generated by the long-running process, we weren't able to use obvious choices as Blackfire.
After a little googling we decided to use php-memprof as a tool to collect data. Sadly there was no expert to take to read KCacheGrind so we dumped data as an array using memprof_dump_array and find by analysing output that memory is building up somewhere in Doctrine.

After good old-fashioned "comment out some code to see if the error disappears" we discovered that the memory was building up due to usage of Doctrine\ORM:Query::getOneOrNullResult in our shared library.
The memory leak was tough so subtle, that it increased only ~20MB on every thousand calls.

After changing getOneOrNullResult to execute memory leak disappeared.

Our application is now working smoothly but the problem in Doctrine still needs to be tracked down.

Write clean & good code - correct caching in PHP

Mati Andreas — Mon, 28 Dec 2020 10:31:42 +0000

Another caching article you think and feel . And who writes PHP these days. But wait, it's not only about caching and PHP, these are only used as examples here.

When writing on another topic a need arose to point out one extra thing in writing a clean and good code: don't invite the wheel.

Consider example

class Foo
{
    private $cache = [];
    private $bar;

    public function __construct(Bar $bar)
    {
        $this->bar = $bar;
    }

    public function getBar(string $someValue): mixed
    {
        if (!key_exists($someValue, $this->cache)) {
            $this->cache[$someValue]= 
                $this->bar->getBySomeValue($someValue);
        }

        return $this->cache[$someValue];
    }
}

It seems very easy to code and test. So let's write it. But what about cache invalidation and memory usage?
So let's add them... by using library. For PHP is Monolog the standard, so
composer install monolog/monolog
and change Foo to

class Foo
{
    private $cacheItemPool;
    private $bar;

    public function __construct(
        CacheItemPoolInterface $cacheItemPool,
        Bar $bar
    ) {
        $this->cacheItemPool = $cacheItemPool;
        $this->bar = $bar;
    }

    public function resolve(string $someValue): mixed
    {
        $cacheKey = urlencode("bar/$someValue");
        $cacheItem = $this->cacheItemPool->getItem($cacheKey);
        if (!$cacheItem->isHit()) {
            $cacheItem->set($this->bar->getBySomeValue($someValue));
            $cacheItem->expiresAfter(new DateInterval('P1D'));
            $this->cacheItemPool->save($cacheItem);
        }

        return $cacheItem->get();
    }
}

Testing doctrine migration in Symfony project

Mati Andreas — Thu, 13 Jun 2019 08:23:20 +0000

Checkout to the master branch
git checkout master.
Update database to match current doctrine schema, also removing excess tables
php bin/console d:s:u --force --complete, where d:s:u is automatic short version of doctrine:schema:update, provided by Symfony console.

NB! It deletes all tables from the database that are not in doctrine schema!
If you want to keep some tables (migration table could also be removed), then run php bin/console d:s:u --dump-sql --complete
do display excess tables, remove them manually and then run php bin/console d:s:u --force to make schema up-to-date with master.
Add all migrationsphp bin/console d:m:v --all --add -n
where d:m:v is automatic short version of doctrine:migrations:version, provided by Symfony console.
Checkout to your feature branch git checkout KEY-123
Run migrations php bin/console d:m:m -n, where d:m:m is automatic short version of doctrine:migrations:migrate, provided by Symfony console.