DEV Community: Alexis Janvier

Handling JWT in Admin Apps the Right Way

Alexis Janvier — Thu, 02 Jul 2020 20:10:56 +0000

The Problem: Safely Storing JWT Tokens in React-Admin

Probably by routine or by Stack Overflow syndrome, we often use a JSON Web Token(JWT) to manage this authentication between our frontend apps and their API. For convenience, we store this token in the browser's localStorage. But this is not a good practice, as Randall Degges explains in his article "Please Stop Using Local Storage". For the most curious, here is an example of how "Stealing JWTs in localStorage via XSS".

But then, how to use a JWT to manage authentication in a more secure way? This blog post will illustrate an implementation of a solution proposed by the Hasura team in their article The Ultimate Guide to handling JWTs on frontend clients. The main concept is to "simply" store this token in memory.

We'll illustrate this with a react-admin application. React-admin has a powerful Authentication Provider subsystem to manage authentication, independent from the rest of the framework code.

But it's not so simple!

The Authentication Route

Let's assume that we have an API with a /authenticate endpoint, which returns a JWT on success. Here is an example implementation of such a server route with Koa and node-jsonwebtoken :

router.post('/authenticate', async (ctx) => {
    const { username, password } = ctx.request.body;

    const user = await getOneByUsername(username);

    if (!user || user.error) {
        ctx.throw(401, user ? user.error : 'Invalid credentials.');
        return;
    }

    if (!bcrypt.compareSync(password, user.password)) {
        ctx.throw(401, 'Invalid credentials.');
        return;
    }

    const token = jwt.sign({ username }, config.security.jwt.secretkey, {
        expiresIn: config.security.jwt.expiration,
    });

    ctx.body = { token };
});

This code is basic. You can find a more complete example in the marmelab/ra-in-memory-jwt demo.

Storing The JWT In Memory

The following frontend code shows a way to store the JWT in memory, rather than in the local storage:

// inMemoryJwt.js
const inMemoryJWTManager = () => {
    let inMemoryJWT = null;

    const getToken = () => inMemoryJWT;

    const setToken = (token) => {
        inMemoryJWT = token;
        return true;
    };

    const ereaseToken = () => {
        inMemoryJWT = null;
        return true;
    }

    return {
        ereaseToken,
        getToken,
        setToken,
    }
};

export default inMemoryJWTManager();

We take advantage of a closure to instantiate the inMemoryJWT variable holds our JWT in memory.

Integrating JWT With The React-Admin AuthProvider

Let's now see how to use this inMemoryJWTManager in a simple React-admin application. First, we declare an App :

// App.js
import React from 'react';
import { Admin, Resource } from 'react-admin';

import myDataProvider from './dataProvider';
import authProvider from './authProvider';
import usersConfiguration from './users';

const dataProvider = myDataProvider('http://localhost:8001/api');
const App = () => (
    <Admin authProvider={authProvider} dataProvider={dataProvider}>
        <Resource name="users" {...usersConfiguration} />
    </Admin>
);

export default App;

Then, we have to configure the React-admin authentication provider:

// in authProvider.js
import inMemoryJWT from './inMemoryJWT';

const authProvider = {
    login: ({ username, password }) => {
        const request = new Request('http://localhost:8001/authenticate', {
            method: 'POST',
            body: JSON.stringify({ username, password }),
            headers: new Headers({ 'Content-Type': 'application/json' })
        });
        return fetch(request)
            .then((response) => {
                if (response.status < 200 || response.status >= 300) {
                    throw new Error(response.statusText);
                }
                return response.json();
            })
            .then(({ token }) => inMemoryJWT.setToken(token));
    },
    logout: () => {
        inMemoryJWT.ereaseToken();
        return Promise.resolve();
    },

    checkAuth: () => {
        return inMemoryJWT.getToken() ? Promise.resolve() : Promise.reject();
    },

    checkError: (error) => {
        const status = error.status;
        if (status === 401 || status === 403) {
            inMemoryJWT.ereaseToken();
            return Promise.reject();
        }
        return Promise.resolve();
    },

    getPermissions: () => {
        return inMemoryJWT.getToken() ? Promise.resolve() : Promise.reject();
    },
};

export default authProvider;

Now, we have to configure the React-admin data provider. Indeed, it's this dataProvider that will be in charge of transmitting the JWT to the API, via an Authorization http header:

// in dataProvider.js
import { fetchUtils } from 'ra-core';
import inMemoryJWT from 'ra-in-memory-jwt';

export default (apiUrl) => {
    const httpClient = (url) => {
        const options = {
            headers: new Headers({ Accept: 'application/json' }),
        };
        const token = inMemoryJWT.getToken();
        if (token) {
            options.headers.set('Authorization', `Bearer ${token}`);
        }

        return fetchUtils.fetchJson(url, options);
    };

    return {
        getList: (resource, params) => {
            const url = `${apiUrl}/${resource}`;
            return httpClient(url).then(({ headers, json }) => {
                return {
                    data: json,
                    total: headers.get('x-total-count'),
                };
            });
        },
        getOne: (resource, params) =>
            httpClient(`${apiUrl}/${resource}/${params.id}`).then(({ json }) => ({
                data: json,
            })),
        getMany: () => Promise.reject(),
        getManyReference: () => Promise.reject(),
        update: () => Promise.reject(),
        updateMany: () => Promise.reject(),
        create: () => Promise.reject(),
        delete: () => Promise.reject(),
        deleteMany: () => Promise.reject(),
    };
};

Our application is already functional and secure. The JWT is no longer visible in the browser's localStorage.

Limitations Of The Memory Storage

But the user experience is not good enough. For instance, when a user reloads the page:

Or when users disconnect from one tab while they're also connected to a second one:

Where do these problems come from, and how can we fix them?

Supporting Several Tabs With Local Storage

When the JWT is stored in the localStorage, two React-admin sessions launched in two tabs will be able to share this JWT. And when user disconnects, deleting the JWT in the localStorage will impact both tabs.

This is no longer the case when the JWT is stored in memory because each instance of React-admin will manage the storage of the JWT independently of each other.

The solution proposed in the article "The Ultimate Guide to handling JWTs on frontend clients" is quite clever, and uses ... the localStorage :)

// inMemoryJwt.js
const inMemoryJWTManager = () => {
    let inMemoryJWT = null;

    // This listener allows to disconnect another session of react-admin started in another tab
    window.addEventListener('storage', (event) => {
        if (event.key === 'ra-logout') {
            inMemoryJWT = null;
        }
    });

    const getToken = () => inMemoryJWT;

    const setToken = (token) => {
        inMemoryJWT = token;
        return true;
    };

    const ereaseToken = () => {
        inMemoryJWT = null;
        window.localStorage.setItem('ra-logout', Date.now());
        return true;
    }

    return {
        ereaseToken,
        getToken,
        setToken,
    }
};

export default inMemoryJWTManager();

Thus, when the user logs out from a tab, the memory manager generates an event on the ra-logout localStorage's item, that is listened by all instances of inMemoryJWT.

Using Short-Lived Tokens For Better Security

An important idea when securing JWTs is to have tokens with a limited lifetime. Let's say 5 minutes. So, if despite all our efforts, this token is stolen, it won't be valid for a very long time. That means that even if the user doesn't reload their page, the session will stop when the token is no longer valid. This implies very short user sessions.

But then, how to keep this level of security without asking users to log in again every five minutes? Well, with a cookie! But here, we'll have to use a very secure cookie (httpOnly, SameSite, etc ...). This cookie will be used to get a new fresh JWT before the current one expires.

First, this will induce additional code in backend to:

Return the token lifetime in addition to the token. We could do this by decoding the token on the frontend side, but that involves useless manipulation!
Set a refresh-token cookie during authentication.
Implement a new /refresh-token endpoint.

The new refresh-token cookie will allow to query the new endpoint /refresh-token. When the frontend queries this route, and in case the refresh-token cookie is still valid, the endpoint should return a new fresh JWT that will replace the outdated one.

I don't detail here the API side implementation, but you can find a more complete example in the demo code of the marmelab/ra-in-memory-jwt repository

But let's see how it'll work on the React-admin application side.

const inMemoryJWTManager = () => {
    let logoutEventName = 'ra-logout';
    let refreshEndpoint = '/refresh-token';
    let inMemoryJWT = null;
    let refreshTimeOutId;

    // This listener allows to disconnect another session of react-admin started in another tab
    window.addEventListener('storage', (event) => {
        if (event.key === logoutEventName) {
            inMemoryJWT = null;
        }
    });

    const setRefreshTokenEndpoint = endpoint => refreshEndpoint = endpoint;

    // This countdown feature is used to renew the JWT in a way that is transparent to the user.
    // before it's no longer valid
    const refreshToken = (delay) => {
        refreshTimeOutId = window.setTimeout(
            getRefreshedToken,
            delay * 1000 - 5000
        ); // Validity period of the token in seconds, minus 5 seconds
    };

    const abordRefreshToken = () => {
        if (refreshTimeOutId) {
            window.clearTimeout(refreshTimeOutId);
        }
    };

    // The method makes a call to the refresh-token endpoint
    // If there is a valid cookie, the endpoint will return a fresh jwt.
    const getRefreshedToken = () => {
        const request = new Request(refreshEndpoint, {
            method: 'GET',
            headers: new Headers({ 'Content-Type': 'application/json' }),
            credentials: 'include',
        });
        return fetch(request)
            .then((response) => {
                if (response.status !== 200) {
                    ereaseToken();
                    global.console.log(
                        'Failed to renew the jwt from the refresh token.'
                    );
                    return { token: null };
                }
                return response.json();
            })
            .then(({ token, tokenExpiry }) => {
                if (token) {
                    setToken(token, tokenExpiry);
                    return true;
                }

                return false;
            });
    };


    const getToken = () => inMemoryJWT;

    const setToken = (token, delay) => {
        inMemoryJWT = token;
        refreshToken(delay);
        return true;
    };

    const ereaseToken = () => {
        inMemoryJWT = null;
        abordRefreshToken();
        window.localStorage.setItem(logoutEventName, Date.now());
        return true;
    }

    const setLogoutEventName = name => logoutEventName = name;

    return {
        ereaseToken,
        getToken,
        setLogoutEventName,
        setRefreshTokenEndpoint,
        setToken,
    }
};

export default inMemoryJWTManager();

//in authProvider.js
//...
const authProvider = {
    login: ({ username, password }) => {
        const request = new Request('http://localhost:8001/authenticate', {
            method: 'POST',
            body: JSON.stringify({ username, password }),
            headers: new Headers({ 'Content-Type': 'application/json' }),
            credentials: 'include',
        });
        inMemoryJWT.setRefreshTokenEndpoint('http://localhost:8001/refresh-token');
        return fetch(request)
            .then((response) => {
                if (response.status < 200 || response.status >= 300) {
                    throw new Error(response.statusText);
                }
                return response.json();
            })
            .then(({ token, tokenExpiry }) => inMemoryJWT.setToken(token, tokenExpiry));
    },
//...

The idea is quite simple: after fetching the JWT, we call refreshToken(), which starts a timeout for another function, getRefreshedToken, which will call the endpoint /refresh-token 5 seconds before the JWT expires (defined in window.setTimeout delay props).

It is thus the cookie's lifetime which will determine the duration of the user's session, rather than the JWT's lifetime.

Persisting Sessions Across Reloads

The code we have just seen allows to have an authenticated session longer than the JWT's lifetime, but it doesn't allow to maintain the session when the page is reloaded.

To reach this result, it should be enough to make a call to the /refresh-token endpoint when checking the user's authorizations (the checkAuth of the authProvider):

//in authProvider.js
//...
    checkAuth: () => {
        console.log('checkAuth');
        if (!inMemoryJWT.getToken()) {
            inMemoryJWT.setRefreshTokenEndpoint('http://localhost:8001/refresh-token');
            return inMemoryJWT.getRefreshedToken().then(tokenHasBeenRefreshed => {
                return tokenHasBeenRefreshed ? Promise.resolve() : Promise.reject();
            });
        } else {
            return Promise.resolve();
        }
    },

Solving Optimistic Rendering Problems

This solution works. But it's not a very satisfactory one:

Due to its optimistic rendering system, React-admin fetches data for the current view (in our example, a list view) before the checkAuth promise returns. So, this fetch will be done without the JWT, and return a 401, leading to a redirection to the authentication page (caused by the checkError method of the authProvider). The login page will "take advantage" of the JWT finally returned and will therefore ... redirect to the original view.

Several solutions are conceivable to solve this problem, according to your needs.

Sometimes some routes don't need JWT tokens. This is the case if you use React-admin to display some views (the list or show view) as public view (without authentication).

But for our example, we consider that all views require authentication. As all endpoints need a token to work, we could implement the fetch to the /refresh-token endpoint directly into the http client:

// in dataProvider

const httpClient = (url) => {
    const options = {
        headers: new Headers({ Accept: 'application/json' }),
    };
    const token = inMemoryJWT.getToken();

    if (token) {
        options.headers.set('Authorization', `Bearer ${token}`);
        return fetchUtils.fetchJson(url, options);
    } else {
        inMemoryJWT.setRefreshTokenEndpoint('http://localhost:8001/refresh-token');
        return inMemoryJWT.getRefreshedToken().then((gotFreshToken) => {
            if (gotFreshToken) {
                options.headers.set('Authorization', `Bearer ${inMemoryJWT.getToken()}`);
            };
            return fetchUtils.fetchJson(url, options);
        });
    }
};

That solves our first problem with the fetch to the list data endpoint that returned a 401. But it brings another problem: as the getPermissions method doesn't have a token, it will trigger a disconnection and then a redirection to login page.

In fact, we have a global race conditions problem between JWT-dependent methods, all these methods may need to make an asynchronous call!

The solution will consist in asking these methods to wait for the end of the possible fetch to the /refresh-token endpoint started by the dataProvider before returning their answer.

That's why a waitForTokenRefresh method must be added to the inMemoryJWTManager.

This method returns a resolved promise if no call to the /refresh-token is in progress. If a call is in progress, it waits for the fetch resolution before returning the resolved promise.

// in inMemoryJWTManager

const inMemoryJWTManager = () => {
    ...
    let isRefreshing = null;

    ...

    const waitForTokenRefresh = () => {
        if (!isRefreshing) {
            return Promise.resolve();
        }
        return isRefreshing.then(() => {
            isRefreshing = null;
            return true;
        });
    }

    const getRefreshedToken = () => {
        const request = new Request(refreshEndpoint, {
            method: 'GET',
            headers: new Headers({ 'Content-Type': 'application/json' }),
            credentials: 'include',
        });

        isRefreshing = fetch(request)
            .then((response) => {
                if (response.status !== 200) {
                    ereaseToken();
                    global.console.log(
                        'Token renewal failure'
                    );
                    return { token: null };
                }
                return response.json();
            })
            .then(({ token, tokenExpiry }) => {
                if (token) {
                    setToken(token, tokenExpiry);
                    return true;
                }
                ereaseToken();
                return false;
            });

        return isRefreshing;
    };

    ...
};

The waitForTokenRefresh method will therefore be used in the authProvider. :

// in authProvider.js
    ...

    checkAuth: () => {
        return inMemoryJWT.waitForTokenRefresh().then(() => {
            return inMemoryJWT.getToken() ? Promise.resolve() : Promise.reject();
        });
    },

    ...

    getPermissions: () => {
        return inMemoryJWT.waitForTokenRefresh().then(() => {
            return inMemoryJWT.getToken() ? Promise.resolve() : Promise.reject();
        });
    },

Logging Out Properly

The last question to be addressed is the logout feature. For the moment, when users log out, as soon as the page reloads, they are connected again!

The only solution is to call a logout route from the API, because the only way to invalidate the refresh cookie is through the API!

// in authProvider.js

    logout: () => {
        const request = new Request('http://localhost:8001/logout', {
            method: 'GET',
            headers: new Headers({ 'Content-Type': 'application/json' }),
            credentials: 'include',
        });
        inMemoryJWT.ereaseToken();

        return fetch(request).then(() => '/login');
    },

Conclusion: Should You Use JWT For Admins?

The code described in this blog post allows to manage a React-admin authentication using a Json Web Token in a secure way without degrading the user experience. The original idea of this method comes from the Hasura team and I built the implementation with the help from my colleagues Julien and Gildas.

But I'm convinced that using JWT in memory isn't always such a good idea.

The use of the code illustrated in this article brings a lot of complexity. This complexity is surely useless in many cases! You must ask yourself the right question: Do I really need to use a Json Web Token to manage my application's authentication (with React-admin or not by the way)?

If you interact for instance with a large system composed of micro-services that use this JWT to share user authentication between different services, then yes, you might want to look into marmelab/ra-in-memory-jwt.

But if your React-admin application interacts with a more monolithic API, chances are your authentication doesn't need more than a cookie. As long as you apply the same good security practices used by the one described in this post managing the /refresh-token endpoint.

A Developer's Diary: Building A Notes Taking App in Shell

Alexis Janvier — Fri, 09 Nov 2018 19:30:30 +0000

Note: This post was originally posted on marmelab.com.

Something I really like about being a developer is that you learn all the time: a pattern, a lib, an obscure configuration trick... In the heat of the action, you're glad, but a few days later, you often forget. It is at these time that you think it would have been a good idea to take notes.

I've already tried some notepads: jrnl, but I've never been able to remember the commands, boostnote that I don't use when I code because it's an extra window, or gist but I can't keep it organized...

And this summer, I received this link "did.txt file" file in my changelog newsletter.

did.txt

Here is how Patrick introduces his blog post:

Goal: create an insanely simple “did” file accessible by terminal

And that's right, it's very simple (it's just adding an alias in your .bash_profile or .zshrc) and bloody effective :

alias did="vim +'normal Go' +'r!date' ~/did.txt"

A did command opens a file into the terminal - so you don't leave your working environment - with the current date. All you have to do is write down this little thing you've just learned.

And I really liked the idea of having a new tool built only with what is already present on the system. But in fact, it's a little too simple. For example, here is what happens if you use did twice on the same day:

Two problems made me think that I would not integrate this command as it stands in my daily routine:

All notes are in a single file, and because did is a daily note-taking tool, this file may become too long to be usable. The point of taking notes is to be able to read them again!
The file is in .txt format, which severely limits the possibilities of formatting, such as code extracts.

This post documents how I customized this good idea to my needs. I tried to keep the same simplicity as the original did and continuing to use only what was already available in the terminal.

One logbook per week

I work in a two-week time box (sprint), so cutting the single file into several weekly logbooks was obvious.

I'll not go into the implementation's details, but show you the (almost) final result. The --help option, man and Google were my friends to get this result.

export DID_PATH=~/.did

function did(){
    export LC_ALL=C
    if [ ! -f ${DID_PATH}/$(date +%Y-%V).txt ]; then
        echo "Week $(date +"%V (%B %Y)") \n\n$(date +"%A %Y-%m-%d")" > ${DID_PATH}/$(date +%Y-%V).txt
    fi
    FILE_EDITION_DATE="$(stat -c "%y" ${DID_PATH}/$(date +%Y-%V).txt)"
    NOW="$(date +"%Y-%m-%d")"
    if [ ${FILE_EDITION_DATE:0:10} != ${NOW} ]; then
        echo "\n$(date +"%A %Y-%m-%d")\n" >> ${DID_PATH}/$(date +%Y-%V).txt
    fi
    unset LC_ALL
    vim +'normal Go' ${DID_PATH}/$(date +%Y-%V).txt
}

Here are the points that seem important to me.

A function rather than an alias: with the introduction of a logic if the log exists, then, else, it was necessary to replace the simple alias by a shell function. if [ ! -f ${DID_PATH}/$(date +%Y-%V).txt ]; then
The date command: it's the command I've tested the most. Here it's simply used to format the current date. For example date +%Y-%V.
The stat command: it allows to retrieve a lot of information about a file, such as the date of the last modification stat -c "%y" ${DID_PATH}/$(date +%Y-%V).txt. This is what allowed me to know if the file had already been edited in the day or not, to decide whether or not to add this date when the logbook file is open.
The terminal locale: the date command is sensitive to the terminal locale. So I had months and days in French. Yep, my system is in french! To be able to keep my notes in English, it was necessary to change the terminal locale during the execution of the command with LC_ALL=C.
The environment variable DID_PATH: this variable is very logical. It simplifies script writing and allows to easily change the storage folder. But it has a great side effect: by using direnv, it will allow you to create a logbook per project!

This new command gets the job done since now it creates one file per week instead of a single file. But this improvement would also be a good example for David Kadavy's article "Complexity is creepy: It’s never just one more thing”.

Indeed, my one more thing brings its share of questions:

With the original did, I always opened the same file. But now did opens the current week's logbook. How will I view my notes from last week?
If I want to open a past logbook, how will I know which ones exist?
With the original did, I could do a search with vim inside my single file. But now, how am I going to find a note through all logbooks?

View a specific logbook : didv (view)

function didv(){
    if [ $1 ]
    then
         cat ${DID_PATH}/${1}.txt
    else
        if [ ! -f ${DID_PATH}/$(date +%Y-%V).txt ]; then
            LC_ALL=C echo "# Week $(date +"%V (%B %Y)") \n\n## $(date +"%A %Y-%m-%d")" > ${DID_PATH}/$(date +%Y-%V).txt
        fi
        cat ${DID_PATH}/$(date +%Y-%V).txt
    fi
}

This function is simpler than did's, but it introduces the use of command arguments: if [ $1 ]. didv opens the current log and didv 2018-32 the log for week 32.

cat is in charge of displaying the file.

List weekly logbooks: didl (list)

I thought that setting up the list of logs would be the fastest feature to set up. I pragmatically tested the ls and tree commands :

But two things bothered me:

I didn't want to display the file extension (for example I want 2018-32 instead of 2018-32.txt),
I wanted to display the month corresponding to the week number to make the list more readable.

Display the month as from the week number with date has been the most complicated part of that did improvement day!

function week2Month(){
    export LC_ALL=C
    year=$(echo $1 | cut -f1 -d-)
    week=$(echo $1 | cut -f2 -d-)
    local dayofweek=1 # 1 for monday
    date -d "$year-01-01 +$(( $week * 7 + 1 - $(date -d "$year-01-04" +%w ) - 3 )) days -2 days + $dayofweek days" +"%B %Y"
    unset LC_ALL
}

function didl(){
    for file in `ls ${DID_PATH}/*.txt | sort -Vr`; do
        filenameRaw="$(basename ${file})"
        filename="${filenameRaw%.*}"
        echo "${filename} ($(week2Month ${filename}))"
    done
}

Search the weekly logbooks: dids (search)

And here we are at the last feature to implement: search the logs. It's grep that is involved.

function dids(){
    export LC_ALL=C
    if [ $1 ]
    then
        for file in `ls ${DID_PATH}/*.txt | sort -Vr`; do
            NB_OCCURENCE="$(grep -c @${1} ${file})"
            if [ ${NB_OCCURENCE} != "0" ]
            then
                filenameRaw="$(basename ${file})"
                filename="${filenameRaw%.*}"
                echo -e "\n\e[32m=> ${filename} ($(week2Month ${filename}), ${NB_OCCURENCE} results) \e[0m" && grep -n -B 1 ${1} ${file}
            fi
        done
    else
         echo "You must add a something to search..."
    fi
    export LC_ALL=C
}

To be able to tag notes and limit the search to these tags, I decided to use a tag's prefix @, allowing to do NB_OCCURENCE="$(grep -c @${1} ${file})". The second use of grep no longer uses this prefix, allowing to display all the lines corresponding to the searched word.

Formatting notes

I was close to the goal! I no longer had one, but 4 commands:

did to open the current logbook on the current date;
didv to view a logbook including the former ones,
didl to list all available logbooks in a readable way,
dids to do a search in all the logs.

Only one point was still pending:

The file is in.txt format, which severely limits the possibilities of formatting, such as code extracts.

A markup language is perfectly adapted for that: markdown.

No luck, there's no basic tool in the terminal to process and display a .md file. However, I had set myself a rule:

"..., and continuing to use only what was already available in the terminal."

It doesn't matter, I'm a punk.

So I found some projects that met the need :

I preferred the vmd rendering. All that remained was to modify all the .txt to .md, add some # and replace cat by vmd in the didv function.

The final scripts

# What did i do today?
# from https://theptrk.com/2018/07/11/did-txt-file/
export MDV_THEME=729.8953
export DID_PATH=~/.did

function did(){
    export LC_ALL=C
    mkdir -p ${DID_PATH}
    if [ ! -f ${DID_PATH}/$(date +%Y-%V).md ]; then
        echo "# Week $(date +"%V (%B %Y)") \n\n## $(date +"%A %Y-%m-%d")" > ${DID_PATH}/$(date +%Y-%V).md
    fi
    FILE_EDITION_DATE="$(stat -c "%y" ${DID_PATH}/$(date +%Y-%V).md)"
    NOW="$(date +"%Y-%m-%d")"
    if [ ${FILE_EDITION_DATE:0:10} != ${NOW} ]
    then
        echo "\n## $(date +"%A %Y-%m-%d")\n" >> ${DID_PATH}/$(date +%Y-%V).md
    fi
    unset LC_ALL
    vim +'normal Go' ${DID_PATH}/$(date +%Y-%V).md
}

function didv(){
    if [ $1 ]
    then
         vmd ${DID_PATH}/${1}.md
    else
        mkdir -p ${DID_PATH}
        if [ ! -f ${DID_PATH}/$(date +%Y-%V).md ]; then
            LC_ALL=C echo "# Week $(date +"%V (%B %Y)") \n\n## $(date +"%A %Y-%m-%d")" > ${DID_PATH}/$(date +%Y-%V).md
        fi
        vmd ${DID_PATH}/$(date +%Y-%V).md
    fi
}

function week2Month(){
    export LC_ALL=C
    year=$(echo $1 | cut -f1 -d-)
    week=$(echo $1 | cut -f2 -d-)
    local dayofweek=1 # 1 for monday
    date -d "$year-01-01 +$(( $week * 7 + 1 - $(date -d "$year-01-04" +%w ) - 3 )) days -2 days + $dayofweek days" +"%B %Y"
    unset LC_ALL
}

function didl(){
    for file in `ls ${DID_PATH}/*.md | sort -Vr`; do
        filenameRaw="$(basename ${file})"
        filename="${filenameRaw%.*}"
        echo "${filename} ($(week2Month ${filename}))"
    done
}

function dids(){
    export LC_ALL=C
    if [ $1 ]
    then
        for file in `ls ${DID_PATH}/*.md | sort -Vr`; do
            NB_OCCURENCE="$(grep -c ${1} ${file})"
            if [ ${NB_OCCURENCE} != "0" ]
            then
                filenameRaw="$(basename ${file})"
                filename="${filenameRaw%.*}"
                echo -e "\n\e[32m=> ${filename} ($(week2Month ${filename}), ${NB_OCCURENCE} results) \e[0m" && grep -n -B 1 ${1} ${file}
            fi
        done
    else
         echo "You must add a something to search..."
    fi
    unset LC_ALL
}

Get it on GitHubGist

Conclusion

I don't know if my scripts can be useful to you. If so, I would be happy to. Otherwise, I would also be happy anyway.

Because it's not the script that's important here. What I would like to have shared in this post is the pleasure of building your own little tool from what is available on your system. It's really very fun! During that day spent modifying the original did.txt, I learned a lot, tested a lot and came up with a result that was exactly what I needed. No more, no less.

It was a bit of a low-dev. I'm very sensitive to low-tech these days.

So I hope this reading has given you some ideas. As far as I'm concerned, I think I'm going to quickly add a didp command.

Did you guess it? p for publishing! Now that I have log books in markdown, it shouldn't be very complicated to publish them on a server, and then add a search engine like Algolia to index them.

Managing State in React: Redux or not Redux?

Alexis Janvier — Fri, 29 Jun 2018 11:48:21 +0000

Note: This post was originally posted on marmelab.com.

At Marmelab we really like to manage the state of React apps using Redux. Its emergence has transformed the way we code our applications: immutability, functional programming, asynchronous API call management with Redux-Saga generators... So much that we sometimes tend to "de facto" integrate Redux into our project start stack.

But is that a good idea? Not sure...

An Example: Managing Meetups With React

Let's take a straightforward meetup management application. It should be able to display:

a list of proposals,
a wish list of talks,
a list of meetup members.

The data comes from a REST API. A login/password protects both the application and the API.

The application is bootstrapped with Create React App and upgraded with:

This is what the project looks like:

The application reflects the typical redux architecture. It starts with an <App /> component that mounts the redux store (<Provider store={store}>) and the router (<ConnectedRouter history={history}>):

// in App.js
...
 export const App = ({ store, history }) => (
    <Provider store={store}>
        <ConnectedRouter history={history}>
            <Container>
                <Header />
                <Switch>
                    <Route exact path="/" component={Home} />
                    <Route path="/talks" component={Talks} />
                    <Route path="/wishes" component={Wishes} />
                    <Route path="/members" component={Members} />
                    <Route path="/login" component={Authentication} />
                    <Route component={NoMatch} />
                </Switch>
            </Container>
        </ConnectedRouter>
    </Provider>
);

Redux users will be comfortable with the file structure that I chose. I grouped all the code related to a feature into a directory. An example with the talks page:

├── talks
│   ├── actions.js
│   ├── reducer.js
│   ├── sagas.js
│   └── Talks.js

The <Talks> page component is a straightforward "connected component":

 // in talks/Talks.js
export const Talks = ({ isLoading, talks }) => (
    <div>
        <h1>Talks</h1>
        {isLoading && <Spinner />}
        {talks && talks.map(talk => <h2 key={talk.id}>{talk.title}</h2>)}
    </div>
);

const mapStateToProps = ({  talks }) => ({
    isLoading: talks.isLoading,
    talks: talks.data,
});

// passing {} as the second's connect argument prevents it to pass dispatch as prop
const mapDispatchToProps = {};

export default connect(mapStateToProps, mapDispatchToProps)(Talks);

The data for the talks is not fetched on componentWillMount, but through a saga listening to route changes:

// in talks/sagas.js
import { put, select, takeLatest } from 'redux-saga/effects';
import { LOCATION_CHANGE } from 'react-router-redux';

import { loadTalks } from './actions';

const hasData = ({ talks }) => !!talks.data;

export function* handleTalksLoading() {
    if (yield select(hasData)) {
        return;
    }

    yield put(loadTalks());
}

export const sagas = function*() {
    yield takeLatest(
        action =>
            action.type === LOCATION_CHANGE &&
            action.payload.pathname === '/talks',
        handleTalksLoading,
    );
};

When the route changes and corresponds to the talks section (action.type === LOCATION_CHANGE && action.payload.pathname === '/talks'), my application triggers an action with the loadTalks function:

// in talks/actions.js
export const LOAD_TALKS = 'LOAD_TALKS';

export const loadTalks = payload => ({
    type: 'LOAD_TALKS',
    payload,
    meta: {
        request: {
            url: '/talks',
        },
    },
});

This action, containing the url to get data for talks inside its meta, will be intercepted by a generic fetch saga action => !!action.meta && action.meta.request:

// in /services/fetch/fetchSagas.js
import { call, put, takeEvery, select } from 'redux-saga/effects';

import { appFetch as fetch } from './fetch';

export const fetchError = (type, error) => ({
    type: `${type}_ERROR`,
    payload: error,
    meta: {
        disconnect: error.code === 401,
    },
});

export const fetchSuccess = (type, response) => ({
    type: `${type}_SUCCESS`,
    payload: response,
});

export function* executeFetchSaga({ type, meta: { request } }) {
    const token = yield select(state => state.authentication.token);
    const { error, response } = yield call(fetch, request, token);
    if (error) {
        yield put(fetchError(type, error));
        return;
    }

    yield put(fetchSuccess(type, response));
}

export const sagas = function*() {
    yield takeEvery(
        action => !!action.meta && action.meta.request,
        executeFetchSaga,
    );
};

Once the fetch is successful, the saga triggers a final action indicating the success of the data recovery (createAction('${type}_SUCCESS')(response)). This action is used by the talks reducer:

// in talks/reducers.js
export const reducer = (state = defaultState, action) => {
    switch (action.type) {
        case LOAD_TALKS:
            return {
                ...state,
                loading: true,
            };
        case LOAD_TALKS_ERROR:
            return {
                ...state,
                loading: false,
                error: action.payload,
            };
        case LOAD_TALKS_SUCCESS:
            return {
                ...state,
                loading: false,
                data: action.payload,
            };
        case LOGOUT:
            return defaultState;
        default:
            return state;
    }
};

It works well. That's pretty smart, even elegant! The use of action's meta allows sharing generic behaviours within the application (data fetching but also error handling or logout).

It's Smart, But It's Complex

It's not easy to find your way around when you discover the application, some behaviours are so magical. To summarize, the app fetch the data with a redux-saga connected to the router, which sends a fetch action intercepted by another generic saga, which in case of success emits another action, action intercepted by the reducer of the page having emitted the very first action of the chain...

Some might say that it's an abusive use of redux, but it's mostly the result of several projects done on this stack, with the experience of rewriting actions and reducers.

Added to this complexity, there is also a significant amount of plumbing, i.e. many files repeated for each feature (actions, reducers and other sagas).

Let's analyse the example application with its three pages, its home and its login page:

 ❯ cloc services/cra_webapp/src
      32 text files.
      32 unique files.
       0 files ignored.

github.com/AlDanial/cloc v 1.74  T=0.06 s (581.6 files/s, 17722.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JavaScript                      31            150              1            819
CSS                              1              0              0              5
-------------------------------------------------------------------------------
SUM:                            32            150              1            824
-------------------------------------------------------------------------------

31 files, 819 lines of code, it's already a lot for a straightforward application. This code could be simplified a little bit, with the risk of making it less generic.

It's certainly time to ask ourselves if Redux is necessary here?

Redux is a predictable state container for JavaScript apps.

But do different parts of the application modify the same data, requiring a predictable state for this data? No, I just need to display data from the API. Are there components buried in the DOM that can modify the data? No, user interactions are pretty limited.

So I probably don't need Redux.

Fetching Data Without Redux

Let's try fetching data without Redux, or more precisely without Redux-Saga (since it is not directly redux' job to perform the data fetching). I could implement all this fetch logic on each page. However, that would be setting up very repetitive mechanics and a lot of duplicated code. So I have to find a generic way to fetch data from the API without introducing too much duplication and complexity.

The render prop pattern is an excellent candidate for this kind of problem!

Let's create a DataProvider component:

// in DataProvider.js
import React, { Component, Fragment } from 'react';
import { Redirect } from 'react-router';
import { appFetch } from './services/fetch';

export class DataProvider extends Component {
    static propTypes = {
        render: PropTypes.func.isRequired,
        url: PropTypes.string.isRequired,
    };

    state = {
        data: undefined,
        error: undefined,
    };

    fetchData = async props => {
        const token = window.sessionStorage.getItem('token');
        try {
            const data = await appFetch({ url }, token);
            this.setState({
                data: data.response,
                error: null,
            });
        } catch (error) {
            this.setState({
                error,
            });
        }
    };

    componentDidMount() {
        return this.fetchData(this.props);
    }

    render() {
        const { data, error } = this.state;
        const { location } = this.props;

        if (error) {
            return error.code >= 401 && error.code <= 403 ? (
                <Redirect to="/login" />
            ) : (
                <p>Erreur lors du chargement des données</p>
            );
        }


        return (
            <Fragment>
                {data ? (
                    <p>Aucune donnée disponible</p>
                ) : (
                    this.props.render({
                        data,
                    })
                )}
            </Fragment>
        );
    }
}

This component fetches data from the prop url during the componentDidMount. It manages error and missing data. If it gets data, it delegates the rendering to the function passed as render prop (this.props.render({ data })).

Let's implement this component on the talk page:

// in talks/Talks.js
import React from 'react';
import PropTypes from 'prop-types';

import { DataProvider } from '../DataProvider';

export const TalksView = ({ talks }) => (
    <div>
        <h1>Talks</h1>
        {talks && talks.map(talk => <h2 key={talk.id}>{talk.title}</h2>)}
    </div>
);

TalksView.propTypes = {
    talks: PropTypes.array,
};

export const Talks = () => (
    <DataProvider
        url="/talks"
        render={({ data }) => <TalksView talks={data} />}
    />
);

I now have two components:

the TalksView component, that only displays data, no matter where it comes from,
the Talks component, using the DataProvider to get the data and TalksView to display it render={({ data }) => <TalksView talks={data} />}.

It's simple, effective and readable!

There is an excellent library implementing this type of DataProvider : react-request: Declarative HTTP requests for React

I am now ready to remove Redux from the application.

Let's relaunch the analysis of our project:

❯ cloc services/cra_webapp/src
      16 text files.
      16 unique files.
       0 files ignored.

github.com/AlDanial/cloc v 1.74  T=0.04 s (418.9 files/s, 13404.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JavaScript                      15             64              1            442
CSS                              1              0              0              5
-------------------------------------------------------------------------------
SUM:                            16             64              1            447
-------------------------------------------------------------------------------

So I went from 819 lines of code to 442 lines, almost half as much. Not bad!

Replacing The Redux Store By React State

In the current state, each page gets data using the DataProvider. However, my application requires authentication to obtain user information through a json-web-token.

How will this user information be transmitted to the individual components without the Redux store? Well, by using the state of the higher level component (App.js), and passing the user as a prop to the child components that need it (PrivateRoute.js, Header.js).

In short, let's make React code again!

// in App.js
import React, { Component } from 'react';
import { BrowserRouter as Router, Route, Switch } from 'react-router-dom';

import { Authentication } from './authentication/Authentication';
import { Header } from './components/Header';
import { PrivateRoute } from './PrivateRoute';
import { Talks } from './talks/Talks';


export class App extends Component {
    state = {
        user: null,
    };

    decodeToken = token => {
        const user = decode(token);
        this.setState({ user });
    };

    componentWillMount() {
        const token = window.sessionStorage.getItem('token');

        if (token) {
            this.decodeToken(token);
        }
    }

    handleNewToken = token => {
        window.sessionStorage.setItem('token', token);
        this.decodeToken(token);
    };

    handleLogout = () => {
        window.sessionStorage.removeItem('token');
        this.setState({ user: null });
    };

    render() {
        const { user } = this.state;
        return (
            <Router>
                <div>
                    <Header user={user} onLogout={this.handleLogout} />
                    <Switch>
                        <PrivateRoute
                            path="/talks"
                            render={() => (
                                <Talks />
                            )}
                            user={user}
                        />
                        <Route
                            path="/login"
                            render={({ location }) => (
                                <Authentication
                                    location={location}
                                    onNewToken={this.handleNewToken}
                                />
                            )}
                        />
                    </Switch>
                </div>
            </Router>
        );
    }
}

Note: I know: storing the token in window.sessionStorage is a bad practice. But this allows me to quickly set up authentication for the sake of this example. This has nothing to do with the removal of Redux.

// in PrivateRoute.js
import React from 'react';
import PropTypes from 'prop-types';
import { Redirect, Route } from 'react-router';

/**
 * This Route will redirect the user to the login page if needed.
 */
export const PrivateRoute = ({ user, ...rest }) =>
    user ? (
        <Route {...rest} />
    ) : (
        <Redirect
            to={{
                pathname: '/login',
                state: { from: rest.location },
            }}
        />
    );

PrivateRoute.propTypes = {
    user: PropTypes.object,
};

// in components/Header.js
import React from 'react';
import PropTypes from 'prop-types';

import { Navigation } from './Navigation';

export const Header = ({ user, onLogout }) => (
    <header>
        <h1>JavaScript Playground: meetups</h1>
        {user && <Navigation onLogout={onLogout} />}
    </header>
);

Header.propTypes = {
    user: PropTypes.object,
    onLogout: PropTypes.func.isRequired,
};

My application being relatively simple, the transmission of the user as a prop to the children is not really a problem.

Let's say I want to make my navigation bar prettier, with a real logout menu displaying the user's name. I'll have to pass this user to the Navigation component.

<Navigation onLogout={onLogout} user={user}/>

Moreover, if the <UserMenu> component uses another component to display the user, I'll have to transmit my user again:

const UserMenu = ({ onLogout, user }) => {
    <div>
        <DisplayUser user={user} />
        <UserSubMenu onLogout={onLogout} />
    </div>
}

The user has been passed through 4 components before being displayed...

What about a more complex and/or heavier application? This can become very painful. It's one of the situations where it becomes legitimate to ask the question of the use of Redux!

However, there is now a straightforward solution to transmit data from one component to others that are deeper in the React tree: the React Context.

Passing The State Down Using React Context

The React.createContext method generates two components:

const {Provider, Consumer} = React.createContext(defaultValue);

a Provider responsible for distributing the data,
a Consumer that's able to read the provider data.

Let's go back to the three previous components.

// in App.js
import React, { Component } from 'react';
import { BrowserRouter as Router, Route, Switch } from 'react-router-dom';
import styled from 'styled-components';
import { decode } from 'jsonwebtoken';

...

const UserContext = React.createContext({
    user: null,
    onLogout: () => true,
});

export const UserConsumer = UserContext.Consumer;
const UserProvider = UserContext.Provider;

export class App extends Component {
    ...

    render() {
        const { user } = this.state;
        return (
            <UserProvider
                value={{
                    user,
                    onLogout: this.handleLogout,
                }}
            >
                <Router>
                    <Container>
                        <Header />
                        <Switch>
                            <PrivateRoute
                                exact
                                path="/"
                                render={({ location }) => (
                                    <Home location={location} />
                                )}
                            />
                        ...

// in PrivateRoute.js
import React from 'react';
import PropTypes from 'prop-types';
import { Redirect, Route } from 'react-router';

import { UserConsumer } from './App';

const PrivateRouteWithoutContext = ({ user, ...rest }) =>
    user ? (
        <Route {...rest} />
    ) : (
        <Redirect
            to={{
                pathname: '/login',
                state: { from: rest.location },
            }}
        />
    );

PrivateRouteWithoutContext.propTypes = {
    user: PropTypes.object,
};

export const PrivateRoute = props => {
    return (
        <UserConsumer>
            {({ user }) => (
                <PrivateRouteWithoutContext user={user} {...props} />
            )}
        </UserConsumer>
    );
};

Note that the Consumer uses the render prop pattern.

// in components/Header.js
import React from 'react';
import PropTypes from 'prop-types';

import { UserConsumer } from '../App';
import { Navigation } from './Navigation';

export const HeaderWithoutContext = ({ user, onLogout }) => (
    <header>
        <h1>JavaScript Playground: meetups</h1>
        {user && <Navigation onLogout={onLogout} />}
    </header>
);

HeaderWithoutContext.propTypes = {
    user: PropTypes.object,
    onLogout: PropTypes.func.isRequired,
};

export const Header = () => {
    return (
        <UserConsumer>
            {({ user, onLogout }) => (
                <HeaderWithoutContext user={user} onLogout={onLogout} />
            )}
        </UserConsumer>
    );
};

React Context is a simple way to teleport data directly from a level N component of the application to any level N-x children component.

So, Redux or Not Redux ?

Redux becomes interesting as soon as a project reaches a certain level of complexity. However, it's rarely a good idea to prejudge the degree of complexity of your code! I prefer to keep things simple to say to myself: "Great! I'm going to make something complex" afterwards. It reminds me of a few years ago, when Symfony was systematically used to start a PHP project, while Silex made it much more comfortable and faster to get started.

Nevertheless, just like Symfony, using Redux can become a very wise choice.

Using it at the beginning of the project is just a premature decision.

It's not really fresh news 😄

You Might Not Need Redux.
— Dan Abramov (@dan_abramov) 19 septembre 2016

Also, beyond these somewhat theoretical considerations, it seems that there are also beneficial effects to the fact of going away from Redux.

First, I focus more on React! By coding the second example in this post, I rediscovered the pleasure of building an application only from bricks of components: it's like playing Lego. The use of render prop allows code re-use throughout the project while maintaining this logic of nesting React components. It is a powerful pattern, less magical than the HOC. Furthermore, it will adapt to the possible implementation of Redux when the time comes. The proof of this is react-admin 2.0 which dissociate the UI part from the application logic, thanks to a render prop.

Finally, this seems the direction taken by the React team. With the new Context API, they offer the possibility to set up a global store easily shareable without adopting Redux.