DEV Community: Andrea Olivato

How you write your String matters: benchmarking PHP

Andrea Olivato — Fri, 09 Jul 2021 16:01:51 +0000

There are a lot of ways in which you can specify, use and print string in PHP. They all have different benefits and drawbacks and that's not the content of this article. The focus of these tests is to determine how using different string specifications affects the performances of your PHP code.

TL;DR: Single quoted strings are the most performing, both for simple text and for concatenation with other variables.

Tests

Compared 3 types of string specification:

Single Quoted (e.g. echo 'string';)
Double Quoted (e.g. echo "string";)
HEREDOC (e.g. echo <<<EOT string EOT;)

For each type of specification, 2 tests:

Simple string (e.g. 'string')
String + a variable (e.g. 'string'.$var)

Iterations

In order to simulate a strong workload and have more data, the code ran 10 iterations with a 5-second pause. For each iteration 1 Million echo were executed, without any pause.

To make sure that the machine load doesn't skew the results, all the 3 different specifications are executed within the same iteration, one after the other, with the same 5-second delay. So at any point in time, the load should be equally distributed among the different specifications.

Buffer

Since the actual printing of the strings must not affect these tests, ob_start() and ob_end_clean() are used to not print anything, and ob_clean() is used within each iteration to free up the buffer.

The Code

This is the code used to run the tests.

<?php

define('MAX_PER_ITERATION', 10000000);
define('MAX_ITERATIONS', 10);
define('SLEEP_TIME', 5);

echo "TEST 1: Simple String\n";
$times = [
    'quote' => [],
    'doublequote' => [],
    'heredoc' => []
];
for ($k = 1; $k <= MAX_ITERATIONS; $k++) {
    echo "Iteration ".$k."\n";

    /* Start Single Quote tests */
    echo "\tQuote\n";
    ob_start();
    $start = microtime(true);
    for ($i = 0; $i < MAX_PER_ITERATION; $i++) {
        echo 'string';
        ob_clean();
    }
    $diff = (microtime(true) - $start);
    ob_end_clean();
    $times['quote'][] = $diff;
    sleep(SLEEP_TIME);

    /* Start Double Quote tests */
    echo "\tDouble Quote\n";
    ob_start();
    $start = microtime(true);
    for ($i = 0; $i < MAX_PER_ITERATION; $i++) {
        echo "string";
        ob_clean();
    }
    $diff = (microtime(true) - $start);
    ob_end_clean();
    $times['doublequote'][] = $diff;
    sleep(SLEEP_TIME);

    /* Start HEREDOC tests */
    echo "\tHeredoc\n";
    ob_start();
    $start = microtime(true);
    for ($i = 0; $i < MAX_PER_ITERATION; $i++) {
        echo <<< EOT
            string
        EOT;
        ob_clean();
    }
    $diff = (microtime(true) - $start);
    ob_end_clean();
    $times['heredoc'][] = $diff;
    sleep(SLEEP_TIME);
}
foreach ($times as $type => $results) {
    echo $type."\n";
    foreach ($results as $r) {
        echo "\t\t".$r."\n";
    }
    $avg = array_sum($results) / sizeof($results);
    echo "\tAvg: ".$avg."\n";
}

sleep(100);

echo "TEST 2: String with Variable\n";
$times = [
    'quote' => [],
    'doublequote' => [],
    'heredoc' => []
];
$variable = 'string2';
for ($k = 1; $k <= MAX_ITERATIONS; $k++) {
    echo "Iteration ".$k."\n";

    /* Start Single Quote tests */
    echo "\tQuote\n";
    ob_start();
    $start = microtime(true);
    for ($i = 0; $i < MAX_PER_ITERATION; $i++) {
        echo 'string'.$variable;
        ob_clean();
    }
    $diff = (microtime(true) - $start);
    ob_end_clean();
    $times['quote'][] = $diff;
    sleep(SLEEP_TIME);

    /* Start Double Quote tests */
    echo "\tDouble Quote\n";
    ob_start();
    $start = microtime(true);
    for ($i = 0; $i < MAX_PER_ITERATION; $i++) {
        echo "string $variable";
        ob_clean();
    }
    $diff = (microtime(true) - $start);
    ob_end_clean();
    $times['doublequote'][] = $diff;
    sleep(SLEEP_TIME);

    /* Start HEREDOC tests */
    echo "\tHeredoc\n";
    ob_start();
    $start = microtime(true);
    for ($i = 0; $i < MAX_PER_ITERATION; $i++) {
        echo <<< EOT
            string {$variable}
        EOT;
        ob_clean();
    }
    $diff = (microtime(true) - $start);
    ob_end_clean();
    $times['heredoc'][] = $diff;
    sleep(SLEEP_TIME);
}
foreach ($times as $type => $results) {
    echo $type."\n";
    foreach ($results as $r) {
        echo "\t\t".$r."\n";
    }
    $avg = array_sum($results) / sizeof($results);
    echo "\tAvg: ".$avg."\n";
}

Summary Results

Unsurprisingly, the Single Quoted strings are the fastest: as they don't require parsing for variables or special characters (e.g. \n), they run quicker than the others both for simple strings and for variables.

Simple String: 0.3838947773 seconds for 1 Million executions
String+Variable: 0.532834816 seconds for 1 Million executions

Heredoc performs a bit slower than Single Quoted strings, but faster than Double Quoted ones, especially when without variables.

Simple String: 0.387871933 seconds for 1 Million executions
String+Variable: 0.553892827 seconds for 1 Million executions

Finally Double Quoted strings:

Simple String: 0.3914105415 seconds for 1 Million executions
String+Variable: 0.5623492718 seconds for 1 Million executions

Complete Results

Below you can see a sneak peak of the final results. If you click on the image or here , you can access a shared spreadsheet with the complete data.

System/Versions

PHP 7.4.19
Amazon Linux 2 AMI
Kernel 4.14.232-177.418.amzn2.x86_64
16GB Memory
4 Virtual CPUs

Fix `ld: library not found for -lgomp` when installing Imagick 3.5 via PHP pecl on Mac

Andrea Olivato — Mon, 21 Jun 2021 06:14:43 +0000

Update 25 July 2021

As of 22 July 2021, version 3.5.1 is stable in the pecl channel and can be used to compile imagick for both PHP 7 and PHP 8.

pecl install imagick

The general command will not fail anymore

TL;DR

Install the older version (3.4.4) of Imagick forcing it via pecl

pecl install imagick-3.4.4

Intro

The last version (3.5) of the PHP extension for Imagemagick ( Imagick ) introduces a compilation error on Apple systems, which is independent from the PHP version (tested on 7.4 and 8) and Imagemagick version (tested on 6.x and 7.x) and is instead related to the missing/broken support for gomp of the gcc compiler.

This is not related to Apple Silicon either, as I've tested on an Intel machine.

Error

The error is thrown while compiling, after running

pecl install imagick

The complete error is

ld: library not found for -lgomp

Potential Fixes

There are several Stack Overflow threads talking about compilation errors due to Gomp. The most comprehensive I've found is this.

I've tried few of the fixes proposed in the answer, but most require compiling a different version of gcc, which would make subsequent updates via homebrew unstable.

Also tried this other amazing thread on Stack Overflow which suggests llvm and revised compiler flags, but it didn't work for this specific case, probably due to built-in parameters in the pecl compilation process.

Actual Fix

In the end the final fix was to downgrade the version of Imagick to 4.3.3, which compiles perfectly.

You can do so either by downloading the release from github or simply by forcing the version in pecl as follows:

pecl install imagick-3.4.4

Retrieve Video List from the official TikTok APIs

Andrea Olivato — Thu, 03 Jun 2021 04:11:11 +0000

The TikTok PHP SDK has been updated, and the new v0.2 brings support for retrieving the User’s Videos.

If you’re new to the TikTok PHP SDK, please read my previous tutorial on how to log in and receive the Access Token. In that article, I also provide a bit of backstory and more general information about the TikTok Login Kit APIs for Web.
Once you’re all caught up and know how to log in and retrieve the access Token, retrieving the Videos is blazing fast.

The easiest way is to use the getUserVideoPages helper method, since it provides built-in pagination.

$videos = $_TK->getUserVideoPages();

This will return an array that contains all the videos of the logged in user. Watch out because this will keep looping and calling the APIs for all the videos you have. So you might want to add the max_pages parameter. For example to limit to the latest 10 pages only:

$videos = $_TK->getUserVideoPages(10);

If you want more control over the pagination, you can use the higher-level getUserVideos method, which allows you to manage your own pagination via the cursor and num_results parameters. For example, let’s say you only want the latest video:

$videos = $_TK->getUserVideos(0, 1);

The pagination info is provided in the cursor and has_more returned values.

Here’s how you can implement your own pagination:

$videos = [];
$cursor = 0;
$count_pages = 0;
$max_pages = 3;
while ($vids = $this->getUserVideos($cursor)) {
    $count_pages++;
    if ($count_pages > $max_pages && $max_pages > 0) {
        break;
    }
    foreach ($vids['videos'] as $v) {
        $videos[] = $v;
    }
    if ($vids['has_more']) {
        $cursor = $vids['cursor'];
    } else {
        break;
    }
}

Once you have your $videos you can use the simple getter methods, for example getCoverImageURL to get all the available parameters.

Here’s a quick sample, populating a table

foreach ($videos as $v) {
   $trs[] = <<<HTML
       <tr>
           <td width="100"><img src="{$v->getCoverImageURL()}" style="width:100%"></td>
           <td width="100">
               <strong>ID</strong>: {$v->getID()}<br /><br />
               <strong>URL</strong>: {$v->getShareURL()}<br /><br />
               <strong>Caption</strong>: {$v->getVideoDescription()}
           </td>
       </tr>
HTML;
  }

All done! If you have any suggestion on the class or its implementation please feel free to reach out on GitHub or Twitter.

How to implement the TikTok Login Kit for Web in PHP

Andrea Olivato — Sun, 30 May 2021 10:11:48 +0000

TikTok has recently released their new API to allow Social Login via their platform. They called it Login Kit for Web.

With more Gen-Z and Millennials in the platform, allowing the simplified signup and login process via TikTok is the logical step for many Web Apps, including our very own Lnk.Bio.

While the implementation is a very basic oauth2 flow, there were no published class nor guides specific to PHP implementation, so I thought useful to publish an Open-source class and this quick tutorial based on such class.

Step 1: Register your App

First of all, you need to register in the TikTok Developers platform. Process is easy and approval is quite fast; it took us just 24hrs to get approved with Lnk.Bio](https://lnk.bio/).

Pay attention to the approved redirect URLs. While the TikTok page calls them “Redirect Domains”, they actually require full URLs, so the full path of the page where you’re redirected after the TikTok login.

After you register your application you will receive, as usual a Client Key and Client Secret.

Step 2: Install helper class

Use Composer to install the PHP helper

composer require gimucco/tiktok-loginkit

If, for any reason, you’re not using Composer, you can download the class from Github. Current stable is v0.1.

Step 3: Full Login Example

The login flow is typical:

Generate a Redirect Url to send the user to TikTok to authorise your app
Receive the Authorisation Token
Exchange the Authorisation Token with an Access Token that you can use for the rest of the calls
Make a sample call to retrieve the User Information

Here’s a quick code sample

session_start(); // Important! Required for STATE Variable check and prevent CSRF attacks
require_once __DIR__.'/../../../autoload.php';
use gimucco\TikTokLoginKit;

$api_key = ''; // Your API Key, as obtained from TikTok Developers portal
$api_secret = ''; // Your API Secret, as obtained from TikTok Developers portal
$redirect_uri = ''; // Where to return after authorization. Must be approved in the TikTok Developers portal
$_TK = new TikTokLoginKit($api_key, $api_secret, $redirect_uri);

if (TikTokLoginKit\Connector::receivingResponse()) { // Check if you're receiving the Authorisation Code
    try {
        $token = $_TK->verifyCode($_GET[TikTokLoginKit\Connector::CODE_PARAM]); // Verify the Authorisation code and get the Access Token

        // ** Your logic to store the access token goes here


        $user = $_TK->getUser(); // Retrieve the User Object

        // ** Your logic to store or use the User Info goes here

        // Print some HTML as example
        echo <<<HTML
        <table width="500">
            <tr>
                <td with="200"><img src="{$user->getAvatarLarger()}"></td>
                <td>
                    <br />
                    <strong>ID</strong>: {$user->getOpenID()}<br /><br />
                    <strong>Name</strong>: {$user->getDisplayName()}
                </td>
            </tr>
        </table>
HTML;
    } catch (Exception $e) {
        echo "Error: ".$e->getMessage();
        echo '<br /><a href="?">Retry</a>';
    }
} else { // Print the initial login button that redirects to TikTok
    echo '<a href="'.$_TK->getRedirect().'">Log in with TikTok</a>';
}

All done! If you have any suggestion on the class or its implementation please feel free to reach out on GitHub or Twitter.

Distinguish between Business and Creators with the Instagram Graph API

Andrea Olivato — Sat, 22 May 2021 08:41:19 +0000

TL;DR: Use Try/Catch on an empty Content Publishing call and look for a specific exception code to differentiate between Business and Creators accounts.

If you use the new Instagram Graph API (not to be confused with the Instagram Basic Display API) you know that these APIs can be used by Instagram Business Accounts and Creators Accounts.

Until recently, the difference between the two didn’t matter as much, since most features are available to both groups. However, when the new Instagram Content Publishing Api was released, Facebook made it only available for Business Accounts, not Creators.

This means that an Instagram Creator can Log In via Facebook and allow the permission for Content Publishing; but when you actually try to publish the content on their page, the API will throw an exception, saying that the user is not allowed to do so.

The main concern we encountered for Lnk.Bio is that we need to immediately inform the user that they won’t be able to use this specific feature. Otherwise they might start scheduling the posts for the future and we wouldn’t actually be able to publish them.

At first, we thought this information would be readily available within the IG User object, returned when you query the Instagram Graph API asking information about the logged user. However this endpoint doesn’t contain any information about the account type (Business/Creator). What makes this even stranger is that instead the Instagram Basic Display API provide this information in the account_type field of the User Object. The returned value can be one of BUSINESS, MEDIA_CREATOR, or PERSONAL, which is exactly what we were looking for. I can’t really find a reason why this is available in the Basic Display API and not the Graph API.

Looking at StackOverflow it looked like the most common approach for the Graph API is to scrape the user profile, since this information is published in the page JSON. But this is of course not allowed by Instagram T&C and is also not a solid solution, as it would be subject to network pitfalls, page errors, and also Instagram anti-crawler policy that would block your IPs after too many requests. So this was a no go.

In the end the best solution we found is to perform an empty POST request simulating the publishing of an Instagram content to the connected account, catch the exception and look for a specific Exception Code 10 in order to identify the type of account.

In order not to actually publish or create anything on the user account, you can perform an empty call (POST method without any POST data) to the /userid/media endpoint. Then catch the FacebookResponseException and look for the code=10.

In PHP code:

public function isBusiness() {
    try {
        $this->conn->post("/".$this->fb_id."/media", [], $this->getToken());
        return true;
    } catch (FacebookResponseException $e) {
        if ($e->getCode() == 10) {
        return false;
    } catch (\Exception $e) {
        return true;
    }
}

As you can see from the code above, all other exceptions are interpreted as Business Account, while only the Exception with code 10 is interpreted as Creator Account.

If you have a better approach to this without performing any call, let me know in the comments!

Websocket and Protobuf integration in Javascript

Andrea Olivato — Fri, 22 Jan 2021 15:32:39 +0000

WebSockets are incredibly performing when you need a continuous dialogue between the Frontend and the Backend of an application.

If you combine them with Protobuf to structure the message exchange, you have a great combination for near-realtime data exchange.

I’ve recently found myself in need to integrate a remote WebSocket with Protobuf directly in the frontend, without using a backend to elaborate the data, in order to be faster and avoid server load.

While the integration was quite easy, I had troubles finding a good documented tutorial from start to end for frontend (no Node.JS or similar), so here we are.

Libraries and Dependancies

I am using the vanilla implementation of WebSocket in Javascript, without any external libraries, so nothing to include there.

We only need a Protobuf implementation in Javascript, and my choice was ProtobufJS.

Easiest way is to use npm to maintain ProtobufJS, or if you prefer you can use a free CDN

<script src="//cdn.rawgit.com/dcodeIO/protobuf.js/6.X.X/dist/protobuf.min.js"></script>

At the time of writing the stable version is 6.10.2 so the complete inclusion is:

<script src="//cdn.rawgit.com/dcodeIO/protobuf.js/6.10.2/dist/protobuf.min.js"></script>

Bottom line, just find the latest version of protobuf.min.js and include it in your page.

Listen to the WebSocket

The WebSocket implementation is pretty straightforward, and you can find more information here.

The most important change compared to online tutorials I found, is that you need to specify the binaryType of the socket, as shown below

socket.binaryType = 'arraybuffer'

Apart from this change, the implementation is easy:

You create the WebSocket
You listen for the connection to open and send the initial message
You keep listening for incoming messages

Here’s the full code for the WebSocket part

// 1. Socket Init
const socket = new WebSocket('wss://remote-service.com/');
socket.binaryType = 'arraybuffer' // Important!

// 2. Listen to Connection opening
socket.addEventListener("open", function (event) {
    console.log("Connection Opened, sending message");
    socket.send('{"message": "HelloWorld!"}');
};

// Listen to Error Events
socket.addEventListener("error", function(err) {
    console.log("error: ", err);
});

// Listen for Connection closure
socket.addEventListener("close", function() {
    console.log("close");
});

// 3. Most Importantly: Listen for received messages
socket.addEventListener('message', function (event) {
     // Protobuf Implementation here, to manage messages
}

Protobuf to decode the message

If you try to console.log the message received from the last Listener, you will receive a base64 encoded binary array.

This is where Protobuf comes in to decode the message and provide the usable message.

To get started you need to create a .proto file which contains the instructions on how to interpret the binary array you receive. If, like me, you’re implementing this for a remote service, they will provide the .proto file, or you can write one yourself based on their specs. The format is pretty straightforward and it looks like this:

message MyMessage{
  required string title= 1;
  required int32 id = 2;
  optional string text = 3;
}

Once you have the .proto file, just save it and place it in a path that can be reached by the WebServer. In my example I’ve saved it as /js/mymessage.proto.

Now that we have the .proto file ready, we can use it to decode the message that is coming to us from the WebSocket. Expanding the code at point 3 above, we have something like this

socket.addEventListener('message', function (event) {
  // I retrieve the Base64 Encoded string
  msg = event.data
  // I transform such string to the typed array needed
  buffer = Uint8Array.from(atob(msg), c => c.charCodeAt(0))

  // Initiate the Protobuf library by opening the .proto file
  protobuf.load("/js/mymessage.proto", function(err, root) {

    // Retrieve the type of message I want to decode from the .proto file
    var MyMessage = root.lookupType("MyMessage");

    // Finally I can decode my message
    var message = MyMessage.decode(buffer);

    // message now contains an object with the properties specified in the .proto file
    console.log(message)   
  });
}

The whole thing

Here’s the complete script, which should give you a good idea at how to implement a remove WebSocket using Protobuf in Javascript

// 1. Socket Init
const socket = new WebSocket('wss://remote-service.com/');
socket.binaryType = 'arraybuffer' // Important!

// 2. Listen to Connection opening
socket.addEventListener("open", function (event) {
    console.log("Connection Opened, sending message");
    socket.send('{"message": "HelloWorld!"}');
};

// Listen to Error Events
socket.addEventListener("error", function(err) {
    console.log("error: ", err);
});

// Listen for Connection closure
socket.addEventListener("close", function() {
    console.log("close");
});

// 3. Most Importantly: Listen for received messages
socket.addEventListener('message', function (event) {
  // I retrieve the Base64 Encoded string
  msg = event.data
  // I transform such string to the typed array needed
  buffer = Uint8Array.from(atob(msg), c => c.charCodeAt(0))

  // Initiate the Protobuf library by opening the .proto file
  protobuf.load("/js/mymessage.proto", function(err, root) {

    // Retrieve the type of message I want to decode from the .proto file
    var MyMessage = root.lookupType("MyMessage");

    // Finally I can decode my message
    var message = MyMessage.decode(buffer);

    // message now contains an object with the properties specified in the .proto file
    console.log(message)   
  });
}

10 Great Homebrew formulas for Web Developers

Andrea Olivato — Sat, 26 Dec 2020 09:43:38 +0000

Homebrew is the first software I Install on any MAC machine. I think it gives me the feeling to be back on my beloved Linux box, with APT or YUM. It's definitely not the same thing, but you know us nostalgic.

Nostalgia aside, I wanted to summarise a collection of favourite formulas that I find super useful for my daily tasks, specifically thinking about a daily developer life.

*EMP (Nginx, Mysql, PHP) Stack

Homebrew allows a very quick deployment of a *EMP stack, which I prefer compared to a container or virtual environment.

brew install nginx php mysql

Configuration files are located in /usr/local/etc/ and is the same as a Linux box.

Formulas are updated very quickly: PHP 8.0 was available on homebrew just a few days after the stable release.

AWS

Who doesn't need to use AWS on a daily basis to optimise/deploy stuff or simply monitor their infrastructure?

brew install awscli

CURL & WGET

Can't live without them.

brew install curl wget

RCLONE

Rclone is incredibly useful to quickly transfer files across different cloud services. Personally, I've used to backups S3 buckers to Google Drive or replicate S3 buckets into Azure and it works splendidly.

brew install rclone

PHP-CS-FIXER

Php-cs-Fixer is a great command-line utility to enforce coding standards. Easily integrated into deploy flows or simply in your favourite editor, in my case Sublime.

brew install php-cs-fixer

Another alternative I tested and liked is phpcbf which you can install via composer

composer global require squizlabs/php_codesniffer

STRIPE

Stripe CLI is incredibly useful for monitoring of activities, frauds, and automated tests/reporting. The Webhook listener is also very handy.

brew install stripe

HTOP

HTOP is a great improvement of top, which I use everywhere, from DEV servers to my own laptop. I know it's not as modern as YTOP but I got used to it and can't get to change.

brew install htop

Install PHP 8 on AWS Amazon Linux 2

Andrea Olivato — Mon, 07 Dec 2020 02:54:39 +0000

PHP 8 is finally here, and it's time to upgrade our DEV environments to start resolving all those Warnings :)

At Lnk.Bio we use the official AWS Amazon Linux 2 AMI, and usually rely on the library Amazon Linux Extras to update/switch php version, but PHP 8 is yet to be included (and I couldn't find any timeline anywhere).

Therefore, I found the best way to install PHP 8 is to rely on Remi's RPM repository which features all major PHP versions for RHEL/Centos (on which Amazon Linux 2 AMI is sort of based).

The steps are quite easy:

Remove current PHP versions
Add Remi's RPM repo for Centos 7
Install PHP 8
Install PHP 8 additional libraries
Paths/Configuration Notes

Please note that this guide is not intended for those who want to run multiple PHP versions at the same time. It will replace the current PHP version with PHP 8.

Let's go

1. Remove current PHP version

If you don't have a formal documentation (you should!) of which additional PHP packages you need for your app, you can get a list by running.

sudo yum list installed | grep php

Once you're confident you can reinstall everything, start by removing old PHP packages.

sudo yum remove php*

2. Add Remi's RPM repo for Centos 7

You will also need EPEL for Centos 7 for dependencies. I guess most people have this already enabled on Amazon Linux 2 AMI, but I am adding it anyway for clarity:

sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
sudo yum install https://rpms.remirepo.net/enterprise/remi-release-7.rpm
sudo yum-config-manager --enable remi-php80

3. Install PHP 8

Now this is quite easy:

sudo yum install php80

Verify PHP is installed correctly by running

php80 -v

4. Install PHP 8 Additional libraries

If you use additional modules/libraries, you can now install them. Naming convention remained the same, but they have the prefix php80-php-{library}

For example, if you use php-fpm you'll run sudo yum install php80-php-fpm; if you need mysqlnd you'll run sudo yum install php80-php-mysqlnd etc...

5. Paths/Configuration notes

Please note that Remi's PHP will be installed under /opt and the binary is php80. Here are a few locations you need to know:

PHP configuration files (php.ini, php-fpm.d etc..): /etc/opt/remi/php80/
PHP logs: /var/opt/remi/php80/log/
PHP Binary: /usr/bin/php80

You can of course create symlinks if you prefer.