DEV Community: Abdelkarim ELAMEL

Interacting with Redis Using Symfony 3.4

Abdelkarim ELAMEL — Thu, 10 Dec 2020 11:15:57 +0000

Hi, and welcome to another blog post related to Symfony 😃

Let’s start with a fact :

In a big project, there is often some data that is not frequently changing and required to be loaded after being logged in. When the number of users and the number of data increases, this may cause some performance issues and might be annoying for end users.

In these situations, many paths of optimizations are available:

Optimizing application code.

Optimizing database (indexes, …)

…

Caching which we are going to focus on in this post.

There are many ways of caching:

Web caching:

It consists of caching a response for a defined amount of time, which can be done on the client-side(i.e: browser cache) or in the server-side. [Details]

Data caching:

Data caching can avoid unnecessary repeated trips back to the database, it can be handy if the data is not changing frequently and the application is querying for the same data all the time. Some databases like MySQL and MariaDB have caching solutions that help speed up the process.

Application caching:

This type of caching is useful when your program uses a method that is very slow, application caching consists of storing the result of the slow method somewhere and use it for the upcoming requests which will speed up the processing time significantly.

Key-Value data caching:

This way of caching relies on the same concept as application cache but it comes with the advantage of not losing data upon reboots or server failure.

Among the popular key-value data caching databases, there is Redis and Memcached which are both in-memory databases.

As you have noticed in the title of this post, we are going to interact with Redis using a Symfony 3.4 application, so let’s get started 😃.

Project setup

Create a blank Symfony project :

composer create-project symfony/framework-standard-edition RedisSymfony3 "3.4.*"

Adding redis-bundle

composer require snc/redis-bundle

After installing the bundle we need to enable it by adding the bundle class in the bundles array in AppKernel class:

$bundles = [
    ....
    new Snc\RedisBundle\SncRedisBundle(),
];

Enabling the bundle is not enough, a little configuration is required in order to communicate with Redis from our application.

For that we need to add below code in our app/config/config.yml file :

snc_redis:
        clients:
            default:
                type: phpredis
                alias: default
                dsn: redis://hostname # (*)
                logging: '%kernel.debug%'

(*): The hostname must be accessible within the network.

We are all set now, we can fetch and store data in Redis.

Communicating with Redis

By installing redis-bundle, we get access to a Redis client class Snc\RedisBundle\Client\Phpredis\Client which contains various methods (get, set, flush, …).

Fetching all Redis keys.

In order to get all the keys stored in Redis, we use the keys method with * as an argument.

Sample Code :


<?php

namespace AppBundle\Controller;

use Snc\RedisBundle\Client\Phpredis\Client;
use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Routing\Annotation\Route;

class DefaultController extends Controller
{
    /** @var  Client */
    private $redisClient;

    public function __construct(Client $client)
    {
        $this->redisClient = $client;
    }

    /**
     * @Route("/keys", name="keys")
     */
    public function indexAction()
    {
        $redisKeys = $this->redisClient->keys('*');
        return $this->json(['keys' => $redisKeys]);
    }
}

Adding key/value pairs in Redis.

For setting a new key/value pair, there is the set method :

set(key, value)

Sample Code :

<?php

namespace AppBundle\Controller;

use Snc\RedisBundle\Client\Phpredis\Client;
use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Routing\Annotation\Route;

class DefaultController extends Controller
{
    /** @var  Client */
    private $redisClient;

    public function __construct(Client $client)
    {
        $this->redisClient = $client;
    }


    /**
     * @param $key
     * @param $value
     *
     * @Route("/create/{key}/{value}", name="create_key", methods={"GET"})
     * @return JsonResponse
     */
    public function createKeyAction($key, $value) {

        $this->redisClient->set($key, $value);
        return $this->json([
            "status" => Response::HTTP_OK
        ]);
    }
}

Deleting a key/value pair from Redis.

For deleting a key, there is a delete method that accepts the key as the argument.

Sample code :

<?php

namespace AppBundle\Controller;

use Snc\RedisBundle\Client\Phpredis\Client;
use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Routing\Annotation\Route;

class DefaultController extends Controller
{
    /** @var  Client */
    private $redisClient;

    public function __construct(Client $client)
    {
        $this->redisClient = $client;
    }

    /**
     * @param $key
     * @Route("/delete/{key}", name="delete_key", methods={"GET"})
     *
     * @return JsonResponse
     */
    public function deleteKeyAction($key) {
        $this->redisClient->delete($key);
        return $this->json([
            "status" => Response::HTTP_OK
        ]);
    }
}

There is also the possibility to remove a set of keys by a pattern like this :

$this->redisClient->delete($this->redisClient->keys($key."*"));

Sample code :

<?php

namespace AppBundle\Controller;

use Snc\RedisBundle\Client\Phpredis\Client;
use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Routing\Annotation\Route;

class DefaultController extends Controller
{
    /** @var  Client */
    private $redisClient;

    public function __construct(Client $client)
    {
        $this->redisClient = $client;
    }

    /**
     * @param $key
     * @Route("/delete/like/{key}", name="delete_like_key", methods={"GET"})
     *
     * @return JsonResponse
     */
    public function deleteKeyLikeAction($key) {
        $this->redisClient->delete($this->redisClient->keys($key."*"));
        return $this->json([
            "status" => Response::HTTP_OK
        ]);
    }
}

That was it for this post, the full code is available here.

Cheers 😄

How to make push notifications without sockets

Abdelkarim ELAMEL — Thu, 10 Dec 2020 11:05:35 +0000

Nowadays, push notifications are a must have feature in every modern web / mobile application.

Real time update notifications, asynchronous/long running task notification are great use cases for this feature. As an IT guy, you’ve probably tried or have implemented this feature in an application. If you haven’t, you’ve certainly asked yourself the following question: how this can be done ?

Answer : There is different ways, each way has its advantages and drawbacks.

First method : XHR Polling

This method consists of making a repetitive HTTP call after certain amount of time in order to retrieve updates.

Advantages: simple to implement / debug, compatible with all browsers and architectures.

Drawbacks: one way communication,inefficient and resource wasting (some calls may return an empty results since there is no update is made).

Second method: SSE events

The second way consists of opening a persistent HTTP connection between the client and the server. When a change is made, the server sends data in which we call a Server Sent Event (SSE) to the client.

Advantages: Native implementation in JS (EventSource), supported reconnection and state reconciliation mechanisms

Drawbacks: One way communication, uses a persistent connection

Third method: Websockets

WebSocket is a full-duplex protocol allowing a bidirectional communication between the server and the client.

Advantages: full duplex communication

Drawbacks: Long lived connections between the server and the client, no support for reconnection and state reconciliation.

The real dilemma is the persistent connections which is not always possible with serverless plateformes and technologies using short lived connection.

So, how can we achieve the same goal with a fancy solution ?

Answer : The Mercure protocol.

What is Mercure ?

Mercure is a protocol allowing to push data updates to web browsers and other HTTP clients in a convenient, fast, reliable and battery-efficient way. It is especially useful to publish real-time updates of resources served through web APIs, to reactive web and mobile apps.

Among the advantages of this protocol :

Native browser support ;
Compatible with all existing servers and can work with old browsers (IE7+) using an EventSource polyfill ;
Built-in connection re-establishment and state reconciliation ;
JWT-based authorization mechanism (securely dispatch an update to some selected subscribers) ;
Message encryption support ;

More details can be found in the official website : https://mercure.rocks/

Key concepts

After this brief introduction, let’s dive into the involved components of this protocol.

The first component is the Topic which is the unit we publish and subscribe to.

The publisher: is responsible of sending updates to the hub, he is also able to securely dispatch update to specific targets.

The subscriber: can be a server/client side application that subscribes to real-time updates from the hub.

The famous Hub: is a server that handles subscription requests and distributes the content to subscribers when the corresponding topics have been updated.

And last but not least, their is the target(s) which can be a subscriber, or a group of subscribers.

Now, after you’ve had an idea about the components, let’s see how they communicate each other.

Publishing :

In order to send a message to a client application, the publisher issues a POST request to the hub which afterwards dispatches the message to the subscriber(s) using a SSE.

The request must be encoded using the application/x-www-form-urlencoded format.

The request body should contain at least the following data :

topic: the name of the topic which will receive the message.
data: contains the content of the message.

In order to dispatch private updates, we can add the topic parameter to the request body which we contain the target(s) allowed to receive the update.

The publisher must present a valid JWT that contains a claim named “mercure”, this claim must contain a “publish” key which is an array of the authorized targets to dispatch to.

VERY IMPORTANT :

The value of “mercure.publish” determines the capabilities of the publisher.

if “mercure.publish”:

is not defined, then the publisher is not allowed to dispatch any update ;
contains an empty array, then the publisher is only allowed to dispatch public updates ;
contains the reserved string * as an array value, then the publisher is authorized to dispatch updates to all targets ;

Subscribing :

The subscriber / client subscribes to the hub URL in order to receive updates by using a GET request that contains the topic names to get updates from.

A subscriber may need to be authorized to receive updates destined to specific targets. To receive these specific updates, the JWS presented by the subscriber must have a claim named mercure with a key named subscribe that contains an array of strings ;

Authorization :

In order to ensure that both publishers / subscribers are authorized for private updates, a JWS (JSON Web Signature) must be provided ;

There are 2 mechanisms to present the JWS to the hub :

Using an Authorization HTTP header :

Used if the publisher / subscriber is not a web browser.
The header contains a string Bearer followed by the JWS.

Using a cookie :

Used if the publisher / subscriber is a web browser.
The browser should send a cookie named mercureAuthorization which contain the JWS.

When using authorization mechanisms, the connection MUST use an encryption layer such as HTTPS;

Reconnection & State Reconciliation

The connection between the subscriber can be lost at any time and the user may not receive notifications about the changes that happened during that time.

To avoid that, the subscriber should send the id of the last received update. This id should be sent from the hub and must be a Global unique identifier (i.e: GUID, UUID, …).

During the reconnection, the subscriber will automatically re-connect to the hub (according to the SSE specifications).

During this phase, the ID should be sent in Last-Event-ID HTTP Header. It can be also provided as a query parameter (with the same name) during the discovery in order to fetch any update dispatched between the initial resource generation by the publisher and the connection to the hub.

If both HTTP Header and the query parameter are provided, the HTTP Header takes the precedence.

Encryption

Relaying on HTTPS as an encryption is not totally secure, since the hub can be managed by a service provider and anyone who has access to the hub can see the content of all messages.

To ensure a total privacy, the message must be encoded by the publisher before sending it to the publisher using Json Web Encryption. The subscriber must have knowledge of this key in order to decrypt the message. The exchange can be done by any relevant mechanism.

A possible way of that is to send an encoded key in the key-set attribute during the discovery.

I hope that it wasn’t borring for you and you’ve got a clear idea about the Mercure protocol.

You can check an example in my github repository where i used a Symfony backend as a publisher and a React.js web app as a subscriber.

How to setup environment variables for a React.js application for multiple environments

Abdelkarim ELAMEL — Thu, 10 Dec 2020 09:41:49 +0000

One build to rule them all

Using environment variables in a client-side application is a bit tricky, but it becomes easy when you know how client-side applications work.

When you are working on a web application (client-side), you will certainly need to communicate with one or many backends to retrieve /send some data.

When you are in an enterprise-scale, the backend URLs will be different and they will depend on the environment where the application is running (dev, staging, ,UAT and production). This is reasonable, isn’t it?

So, how can we define the values depending on the environment?

The short answer: In many different ways.

I’ll be brief in the first 2 methods and you will know why in a minute :)

Method 1: Using dotenv module.

This approach consists of using files prefixed with ‘.env’ containing environment variables in a key=value representation.

The content of a file will look like this:

REACT_APP_API_URL=https://path/to/my/backend/api

Each environment has its own specific file :

.env : for production environment
.env.development : for development environment

In order to retrieve the value of the variable in your application’s code, you can find the value in the process.env global variable, like this :

const App = () => <h1>{process.env.REACT_APP_API_URL}</h1>;

If you are using CRA (Create React App), you will find all the details about this is the official documentation since it uses the dotenv module internally.

NOTE: You will have to install the dependency if you are not using CRA.

Method 2: Using npm scripts and Webpack.

If you are not using CRA, which does some “MAGIC” behind the scenes, you will have to do that “MAGIC” by yourself ;)

Nothing to be afraid of, CRA uses webpack behind the scenes to bundle your application, it replaces the process.env.REACT_APP_API_URL with the value in the .env file depending on the environment specified for the build.

You can do the same thing with a small chunk of code.

First, you’ll have to install webpack and webpack-cli from npm :

npm install --save-dev webpack webpack-cli

After that, go to your package.json and set your environment variable for each npm script you have.

I suppose that you know Webpack, so I won’t go into the details of the config files of each environment.

It’s not over yet !!

The application will not work since we are not telling webpack how the handle the process.env statements in our code.

const App = () => <h1>{process.env.REACT_APP_API_URL}</h1>;

Inside your webpack configuration file, you will need to add an entry in the plugins list which will interpolate the expression and places the actual value of the environment variable.

Now, you will be able to see the value printed on your web page.

Method 3: Setting values in the deployment phase.

Before going into the details, I would like to mention an issue with the previous methods.

The issue is that you will have to build the application for every environment that you have since the values are injected at the build time.

This is not ideal when you are dealing with multiple environments and you will have to store each build for each environment somewhere making it hard to manage (in a cloud and container point of view).

The idea here is the have one “generic” build which isn’t related to any environment and at the deployment phase, the environment-specific values will be injected into the application.

Let’s see this in action :

First of all, we will need a shell script that will be responsible for extracting the environment variables values and injecting them into a javascript file.

This script is reading all the environment variables starting with REACT_APP in the machine (server). It writes them in a .env file which is transformed into a JSON object placed in the env-config.js in the window._env_ object that is accessible from the browser.

#.env.sh

#!/bin/bash

#generate a .env file containing your environment variables
env >> .env

# Recreate config file
rm -rf ./env-config.js
touch ./env-config.js

# Add assignment 
echo "window._env_ = {" >> ./env-config.js

# Read each line in .env file
# Each line represents key=value pairs
while read -r line || [[ -n "$line" ]];
do
  # Split env variables by character `=`
  if printf '%s\n' "$line" | grep -q -e '='; then
    varname=$(printf '%s\n' "$line" | sed -e 's/=.*//')
    varvalue=$(printf '%s\n' "$line" | sed -e 's/^[^=]*=//')
  fi

  # Read value of current variable if exists as Environment variable
  value=$(printf '%s\n' "${!varname}")
  # Otherwise use value from .env file
  [[ -z $value ]] && value=${varvalue}

  # Append configuration property to JS file
  echo "  $varname: \"$value\"," >> ./env-config.js
done < .env

echo "}" >> ./env-config.js

Now you will need to place the env-config.js in the public folder of your application and import it in your index.html (You can make another shell script that does this for you if you want to automate stuff)

<!-- index.html -->
<script src="%PUBLIC_URL%/env-config.js"></script>

Accessing your environment variables will be like this :

const App = () => <h1>{window._env_.REACT_APP_API_URL}</h1>;

And that’s it, you are all set now!

I hope that you’ve liked the post.

Until next time, I would like to say: Stay Safe, Stay at home, and Keep coding.

Cheers.

[Symfony]: Empty token storage when injecting service in an event listener

Abdelkarim ELAMEL — Thu, 10 Dec 2020 09:38:22 +0000

Another Symfony mistery

You might have been struggling to fetch the logged-in user from a listener that includes a UserService class which contains the logic of fetching the logged-in user.

Let's take the below code as an example:

//UserService.php

<?php

namespace App\Service;
use Symfony\Component\Security\Core\Authentication\Token\Storage\TokenStorageInterface;


class UserService {

  private $tokenStorage;

  private $currentUser;

  /**
  * @required
  */
  public function setSecurityContext(TokenStorageInterface $tokenStorage) {
    $token = $tokenStorage->getToken();
    if ($token) {
      $this->currentUser = $token->getUser();
    }
  }

  public function getCurrentUser() {
    return $this->currentUser;
  }
}

?>

//MyListner

<?php
namespace App\Listener;
use Doctrine\Persistence\Event\LifecycleEventArgs;

class MyListener {

  private $userService;

  public function __construct(UserService $userService) {
    $this->userService = $userService;
  }

  public function onFlush(LifecycleEventArgs $args) {

    $currentUser = $this->userService->getCurrentUser();

    var_dump($currentUser); // prints NULL
  }

}
?>

Why the currentUser is NULL ?

The answer: Doctrine listeners are called whenever a corresponding event occurs. Initialization of them may occur before the security context.

You have two options here:

The first one: inject your TokenStorage directly in your Listener and retrieve the token inside your event handler.

//MyListner

<?php
namespace App\Listener;
use Doctrine\Persistence\Event\LifecycleEventArgs;
use Symfony\Component\Security\Core\Authentication\Token\Storage\TokenStorageInterface;


class MyListener {

  private $tokenStorage;

  public function __construct(TokenStorageInterface $tokenStorage) {
    $this->tokenStorage = $tokenStorage;
  }

  public function onFlush(LifecycleEventArgs $args) {

    $currentUser = $this->tokenStorage->getToken()->getUser();

    var_dump($currentUser); // prints a UserInterface object
  }

}
?>

The second one: Fetch the token inside the getCurrentUser method.

//UserService.php

<?php

namespace App\Service;

use Symfony\Component\Security\Core\Authentication\Token\Storage\TokenStorageInterface;

class UserService {

  private $tokenStorage;

  private $currentUser;

  public function __construct(TokenStorageInterface $tokenStorage) {
    $this->tokenStorage = $tokenStorage;
  }


  public function getCurrentUser(TokenStorageInterface $tokenStorage) {
    $token = $tokenStorage->getToken();
    if ($token && is_null($this->currentUser)) {
      $this->currentUser = $token->getUser();
    }

    return $this->currentUser;
  }

}

?>

The hidden trick is that you have to call the getToken method in order to get a fresh token.

That was it, I hope this blog post was helpful.

Cheers