DEV Community: Rowinson Gallego

Making a self-hosted Web Push server 2300% faster

Rowinson Gallego — Sat, 28 Aug 2021 10:57:15 +0000

When I started writting Perfecty Push Notifications for WordPress, a self-hosted Push Server in PHP using wep-push-php, I decided to target almost every WordPress server in the planet (Shared hosts, VPS, Dedicated hosts). It was important to offer a plugin that worked out-of-the-box in almost any installation while at the same time gave a good user experience.

Although saying a 2300% performance improvement is extravagant, you can say "yeah, it was a bad design since the beginning, that's why", I'll argue the reasons behind it and invite you to read the whole post, maybe there will be some take aways for you, specially if you like to build side projects as I do :)

Everything started with the end-user in mind

Because sending thousands of notifications via a web form that waits for minutes in a never loading page while doing the whole processing is awful, I decided to use background jobs to send the notifications.

There are sophisticated ways for background processing in WordPress like Action Scheduler used in Woocommerce, which has automatic adjustments like:

[...] Action Scheduler will only process actions in a request until:

90% of available memory is used

processing another 3 actions would exceed 30 seconds of total request time, based on the average processing time for the current batch

in a single concurrent queue

However, it was particularly problematic in my case because the websites would need that the cron system can reach themselves from the internet, which in some server configurations it's not always granted by default.

Considering that WordPress has its own cron system WP-Cron and is well supported in the vast majority of installations, I decided to start with this one instead. It is built-in and has tons of documentation resources and help available online for the future end-users. The only concern was how it relies on the web traffic to trigger the jobs, however it can be adjusted so that wp-cron.php is executed by a system cron, so it was not a big problem.

I implemented a simple background processing in batches with the default parameters defined as low so that the plugin worked out-of-the-box in almost any installation. The drawback was that it was not fast enough for highly demanding websites since the beginning, however I planned to solve it in future iterations because the important matter was to have an MVP that showcased its value.

Highlights of the original mechanism:

The batch_size setting defined the total number of notifications sent in a WP-Cron execution. It was adjustable and by default it was 30 notifications per job execution.
After sending a batch_size number of notifications, it would auto-schedule himself to send the next batch of notifications.
Used the default batchSize = 1000 parameter from wep-push-php, however the batch_size setting from Perfecty Push would still limit it.

The code at that time looked like this:

    public static function execute_broadcast_batch( $notification_id ) {
        Log::info( 'Executing batch for job id=' . $notification_id );

        $notification = Perfecty_Push_Lib_Db::get_notification( $notification_id );
        if ( ! $notification ) {
            Log::error( "Notification job $notification_id was not found" );
            return false;
        }

        // if it has been taken but not released, that means a wrong state
        if ( $notification->is_taken ) {
            Log::error( 'Halted, notification job already taken, notification_id: ' . $notification_id );
            return false;
        }

        // we check if it's a valid status
        // we only process running or scheduled jobs
        if ( $notification->status !== Perfecty_Push_Lib_Db::NOTIFICATIONS_STATUS_SCHEDULED &&
        $notification->status !== Perfecty_Push_Lib_Db::NOTIFICATIONS_STATUS_RUNNING ) {
            Log::error( 'Halted, received a job with an invalid status (' . $notification->status . '), notification_id: ' . $notification_id );
            return false;
        }

        // this is the first time we get here so we mark it as running
        if ( $notification->status == Perfecty_Push_Lib_Db::NOTIFICATIONS_STATUS_SCHEDULED ) {
            Log::info( 'Marking job id=' . $notification_id . ' as running' );
            Perfecty_Push_Lib_Db::mark_notification_running( $notification_id );
        }

        Perfecty_Push_Lib_Db::take_notification( $notification_id );

        // we get the next batch, starting from $last_cursor we take $batch_size elements
        // we only fetch the active users (only_active)
        $users = Perfecty_Push_Lib_Db::get_users( $notification->last_cursor, $notification->batch_size, 'created_at', 'desc' );

        if ( count( $users ) == 0 ) {
            Log::info( 'Job id=' . $notification_id . ' completed, released' );
            $result = Perfecty_Push_Lib_Db::mark_notification_completed_untake( $notification_id );
            if ( ! $result ) {
                Log::error( "Could not mark the notification job $notification_id as completed" );
                return false;
            }
            return true;
        }

        // we send one batch
        $result = self::send_notification( $notification->payload, $users );
        if ( is_array( $result ) ) {
            $notification                    = Perfecty_Push_Lib_Db::get_notification( $notification_id );
            $total_batch                     = $result[0];
            $succeeded                       = $result[1];
            $notification->last_cursor      += $total_batch;
            $notification->succeeded        += $succeeded;
            $notification->is_taken          = 0;
            $notification->last_execution_at = current_time( 'mysql', 1 );
            $result                          = Perfecty_Push_Lib_Db::update_notification( $notification );

            Log::info( 'Notification batch for id=' . $notification_id . ' sent. Cursor: ' . $notification->last_cursor . ', Total: ' . $total_batch . ', Succeeded: ' . $succeeded );
            if ( ! $result ) {
                Log::error( 'Could not update the notification after sending one batch' );
                return false;
            }
        } else {
            Log::error( 'Error executing one batch for id=' . $notification_id . ', result: ' . $result );
            Perfecty_Push_Lib_Db::mark_notification_failed( $notification_id );
            Perfecty_Push_Lib_Db::untake_notification( $notification_id );
            return false;
        }

        // execute the next batch
        if ( ! wp_next_scheduled( self::BROADCAST_HOOK, array( $notification_id ) ) ) {
            $result = wp_schedule_single_event( time(), self::BROADCAST_HOOK, array( $notification_id ) );
            Log::info( 'Scheduling next batch for id=' . $notification_id . ' . Result: ' . $result );
        } else {
            Log::warning( "Don't schedule next batch, it's already scheduled, id=" . $notification_id );
        }
        return true;
    }

Good for a first version, but problematic

However, the above code is slow and there was a problem with how the batchSize parameter from the web-push-lib worked during my initial tests. This parameter defines the batches size during flushing, by making asynchronous HTTP requests. You can see those batches as concurrent requests, and they can potentially create high spikes in memory and CPU usage, which can cause some weird errors in the downstream components like:

[14-Dec-2020 19:41:36] WARNING: [pool website.com]
child 4593 said into stderr: "PHP message: ERROR |
Failed to send one notification, error:
cURL error 60: Issuer certificate is invalid.
(see https://curl.haxx.se/libcurl/c/libcurl-errors.html) 
for https://fcm.googleapis.com/fcm/send/XXXXXXXXXXXXXXXXXXXXXXX"

Although I initially suspected problems with cURL and the certificates, it was my server that couldn't handle more than 300 notifications concurrently with such a specs. So, instead of tweaking the batchSize parameter from the web-push-php lib, I adjusted the batch_size value in my plugin and never used a value higher than 250 in my production-like environment.

For a fresh installation of the plugin, the default value of batch_size = 30 had a very decent throughput, it took around 3 minutes to send 1.000 notifications, acceptable if you want to send Push Notifications for free. So I launched it:

Launching Perfecty Push Notifications

Rowinson Gallego ・ Dec 30 '20 ・ 6 min read

#wordpress #php #sideprojects #javascript

However, as some websites started to have more than 10.000 users, the plugin was taking more than 23 minutes to complete the batch processing and it was noticeable slow. I needed to do something...

2300% faster

Recently I published the v1.4.0 version with performance improvements that make the plugin 2300% faster. The plugin can now send more than 13.000 notifications in 56 seconds, in a very basic server of 2Gb RAM and 2vCPU, which is a huge gain compared to the 23 minutes it was taking before, or 2300% faster.

The server load after the improvements looked like this:

Highlights of the new mechanism:

The execution of multiple batches is done in a single cron job (before it used multiple cron jobs), which reduced the wasted time between cron job executions (~5s to 10s). At the same time, if it can send all notifications at once, it will do it.
The execution is split in subsequent cron jobs if it is taking more than 80% of the maximum execution time. This can be avoided if the script has no time limit or the limit is very high.
The batch_size setting now defaults to 1.500, before it was 30 and caused weird issues with values higher than 250. With this mechanism I've used values like 20.000 and it works smoothly :)
The parallel flushing (batchSize from web-push-php) is now adjustable and defaults to a low value (50) to avoid the weird cURL issues mentioned above. It can be increased by using better server specs.

If you want to take a look at the code, it's this:

    public static function execute_broadcast_batch( $notification_id ) {
        Log::info( 'Executing batch for job id=' . $notification_id );

        $notification = Perfecty_Push_Lib_Db::get_notification( $notification_id );
        if ( ! $notification ) {
            Log::error( "Notification job $notification_id was not found" );
            return false;
        }

        // if it has been taken but not released, that means a wrong state
        if ( $notification->is_taken ) {
            Log::error( 'Halted, notification job already taken, notification_id: ' . $notification_id );
            return false;
        }

        // we check if it's a valid status
        // we only process running or scheduled jobs
        if ( $notification->status !== Perfecty_Push_Lib_Db::NOTIFICATIONS_STATUS_SCHEDULED &&
        $notification->status !== Perfecty_Push_Lib_Db::NOTIFICATIONS_STATUS_RUNNING ) {
            Log::error( 'Halted, received a job with an invalid status (' . $notification->status . '), notification_id: ' . $notification_id );
            return false;
        }

        // this is the first time we get here so we mark it as running
        if ( $notification->status == Perfecty_Push_Lib_Db::NOTIFICATIONS_STATUS_SCHEDULED ) {
            Log::info( 'Marking job id=' . $notification_id . ' as running' );
            Perfecty_Push_Lib_Db::mark_notification_running( $notification_id );
        }

        Perfecty_Push_Lib_Db::take_notification( $notification_id );

        // we get the next batch, starting from $last_cursor we take $batch_size elements
        // we only fetch the active users (only_active)
        $total_succeeded = 0;
        $cursor          = $notification->last_cursor;
        $start_time      = microtime( true );
        while ( true ) {
            $users  = Perfecty_Push_Lib_Db::get_users( $cursor, $notification->batch_size );
            $count  = count( $users );
            $cursor = $cursor + $count;

            if ( $count == 0 ) {
                Log::info( 'Job id=' . $notification_id . ' completed, released' );
                $result = Perfecty_Push_Lib_Db::mark_notification_completed_untake( $notification_id );
                if ( ! $result ) {
                    Log::error( "Could not mark the notification job $notification_id as completed" );
                    break;
                }
                break;
            }

            $succeeded = self::send_notification( $notification->payload, $users );
            if ( $succeeded !== 0 ) {
                Log::info( "Completed batch, successful: $succeeded, cursor: $cursor" );
                $total_succeeded += $succeeded;
            } else {
                Log::error( 'Error executing one batch for id=' . $notification_id );
                Perfecty_Push_Lib_Db::mark_notification_failed( $notification_id );
                Perfecty_Push_Lib_Db::untake_notification( $notification_id );
                break;
            }

            // check that we don't exceed 80% of max_execution_time
            // in case we do, we split the execution to a next cron cycle to avoid the termination of the script
            // if max_execution_time=0, we never split
            if ( self::time_limit_exceeded( $start_time ) ) {
                Log::warning( 'Time execution is reaching 80% of max_execution_time, moving to next cycle' );
                break;
            }
        }

        if ( $total_succeeded != 0 ) {
            $notification                    = Perfecty_Push_Lib_Db::get_notification( $notification_id );
            $notification->last_cursor       = $cursor;
            $notification->succeeded        += $total_succeeded;
            $notification->is_taken          = 0;
            $notification->last_execution_at = current_time( 'mysql', 1 );
            $result                          = Perfecty_Push_Lib_Db::update_notification( $notification );

            Log::info( 'Notification cycle for id=' . $notification_id . ' sent. Cursor: ' . $notification->last_cursor . ', Succeeded: ' . $total_succeeded );
            if ( ! $result ) {
                Log::error( 'Could not update the notification after sending one batch' );
                return false;
            }

            if ( $notification->status === Perfecty_Push_Lib_Db::NOTIFICATIONS_STATUS_RUNNING ) {
                // execute the next batch
                if ( ! wp_next_scheduled( self::BROADCAST_HOOK, array( $notification_id ) ) ) {
                    $result = wp_schedule_single_event( time(), self::BROADCAST_HOOK, array( $notification_id ) );
                    Log::info( 'Scheduling next batch for id=' . $notification_id . ' . Result: ' . $result );
                } else {
                    Log::warning( "Don't schedule next batch, it's already scheduled, id=" . $notification_id );
                }
            }
        }

        return true;
    }

The process to adjust the performance of the plugin in a WordPress site with this new mechanism is very well described in the official Perfecty Push documentation here: https://docs.perfecty.org/wp/performance-improvements/

Conclusion

Of course, this number can be lowered down much more by adjusting the web server limits (memory limit or the fpm params), or increasing the server specs, or moving away other components if they reside in the same server (mail server, metrics server, external admin panel, etc). The idea is that with the new approach, it's easier for the end users to tune it and achieve a much faster push server.

It also demonstrates that working in iterations helps in showing the product value to the end-users since the beginning, and that it's preferred to have a good working version released on time rather than getting stuck forever until it reaches the absolute perfection. Please understand, perfection takes time and a couple of iterations.

Photos

Photo by Jean Gerber on Unsplash

Photo by ray sangga kusuma on Unsplash

Photo by Josh Calabrese on Unsplash

Hands-on experience with an Open Source project

Rowinson Gallego — Mon, 23 Aug 2021 00:24:59 +0000

After a reflection about my past experiences building side projects, I decided that all my new personal projects would be Open Source, and right after that I created Perfecty Push Notifications, a plugin for WordPress that is a self-hosted alternative to OneSignal or WonderPush for sending Web Push Notifications for free.

Perfecty Push Notification was launched in December 2020. It is a self-hosted Web Push server written in PHP for WordPress websites. So far, it has reached around 3,713 installations with more than 400 websites using it actively. The latest version is 1.3.3, and the next release will include a performance improvement that sends 10.000 notifications/minute in a basic server with 2GB RAM and 1 vCPU.

In this post I'll share my experience with this Open Source project and the things I've learnt from it.

Running lean

When you build a side project with an idea in mind you have two options: build it privately to launch it whenever you're done, or make it public since the beginning. I got inspired by the Running Lean book by Ash Maurya, and I was aware of the potential pitfalls of not closing the feedback loop with your end users from the beginning. You face the risk of not completing the project or launching something that nobody needs.

Building an Open Source project that is open from the beginning helps you exactly in the Problem/Solution fit: the feedback loop is closed as soon as you get the first installs and you're enabled to do continuos iterations using your users' feedback. Of course, for this, you need to follow CI/CD practices, do constant deployments, have good automated tests, an so on... however the details about Running Lean I won't describe them here but you can found them in Mauryas' book. Also, please note that the book is not about Open Source projects but can be applied to them too.

Picture taken from Leanstack

Community traction and support

One of the biggest differences with creating a personal project that only you have access to, is that an Open Source project can be public by design and everything will be under the public scrutiny. Of course this can be seen as a pitfall because it adds you some pressure, but that's exactly what you need to get the things done with your project, and as a side effect the reward is incredible.

In my case, right after publishing Perfecty Push to the WordPress plugin directory, the project gained community traction and I received tens of private messages from many people around the globe asking for more information, quoting the development of something similar, etc. Those messages I cannot share them, but here are some of the public comments the plugin received:

I cannot enjoy more the excitement of this guy talking about Perfecty Push (I loved it!)

However, the most powerful effect is that many people believe in your project and get involved in its development, and that speeds the things up, but at the same time you're now in charge of providing guidance and ultimately deciding on important matters for the project. This gives you a great power, and a great responsibility.

Great powers

You have great powers starting with the first user that decides to install your plugin, and with the first Open Source contributor that wants to join your adventure.

With your users, you have the freedom to define the path of the product features and how it will handle the users' information. You're now in charge of defining how their system will behave with the data they own. This might sound scary, but at the same time demonstrates that your users believe in your idea, and that at the end of the day you were not 100% crazy (maybe 99.999%).

With the contributors you have the ultimate voice in important matters. When no consensus is achieved, you have the power to say if something makes it to reality or not. It doesn't mean that you're the ruler of a tyranny, but it means that when there's not a clear path, you need to take one of the pills, either red or blue, because at this stage you're the one who takes the decision. I imagine in more mature projects this doesn't happen because they require a more complex structure to operate.

Great responsibility

Imagine you're the creator of a JS library that is distributed through npm and is used by hundreds of projects. What happens if by mistake you introduce a serious bug that could take down an app? Those bugs could make their way to production and affect real websites that are run by real businesses. Well, the consequences are similar if by mistake you introduce a serious bug in a WordPress plugin. You can literally take down thousands of websites by mistake.

You have a special responsibility to take care of your end user's platforms, they decided to trust you and believed in your project. They have great expectations from you now. As we're humans, we're error prone, and we need to mitigate these errors with a good testing strategy and extra attention when we publish a new version of our plugin: add automated tests, test it locally, test it in a real production scenario yourself, test it twice, check your email to see if there's something strange around your project, pray, etc.

Disclosure

As of today, I've caused two disruptions that I caught early myself and corrected even before getting any support ticket:

One source file was not included during the build stage in the pipeline and thus, it created an unrecoverable error. Fortunately, WordPress detects these situations and reverts the plugin upgrade in your user's websites. I corrected this almost immediately and the rest is a happy story.
The minified JS SDK version had a mismatched reference in the main project after migrating the repos from my personal Github account to a .org account. This was corrected and didn't have any major consequences, except the users couldn't enjoy the new features included in that version.

Again, this demonstrates that we and the systems we build are error prone, and we need to be humble and do our best to mitigate them.

Ideas, ideas everywhere

One of the most enjoyable parts that can be also stressful, is that everyone wants their opinion to be heard. Everybody wants to influence on the project features, and because you get a massive audience, and maybe you were not prepared for this, you have now the challenge to achieve a balance between hearing the user's voice and making reasonable decisions for the project. This might put you in some difficult situations where you have to say no to a stubborn user, but at the same time you need to find the right words to do it so that he can come back to you afterwards.

It's also important that you take care of your project and make your best effort to answer your users' tickets or emails (they love to receive an answer from you!) Watch out: At the end of the day, in the Problem/Solution fit, this is one of the most important parts because it's the instantiation of the feedback loop abstract.

Conclusion

I really recommend taking the Open Source approach for side projects that have a business idea in mind. If you've created more than three projects in a row that didn't make it to reality, please consider this model and let me know your thoughts!

Photos

Cover: Photo by Sandie Clarke on Unsplash

Photo by Tim Johnson on Unsplash

Photo by Mukuko Studio on Unsplash

Launching Perfecty Push Notifications

Rowinson Gallego — Wed, 30 Dec 2020 14:10:09 +0000

Recently, I wrote a post describing the review process for publishing a plugin in the WordPress Plugin Directory:

Sending a plugin to the Wordpress Plugin Directory

Rowinson Gallego ・ Dec 22 '20 ・ 4 min read

#wordpress #plugins #php #pushnotifications

I've created Perfecty Push Notifications, a WordPress plugin for sending Push notifications using Push API from your server for free. You can check the Github repo and contribute ⚡️. In this post, I will describe what happened next in the review, and how the publishing of the plugin looked like.

Getting the approval

Well, after addressing the initial suggestions from the WordPress plugin review team, I received another email suggesting the upgrade of the plugin dependencies:

At that time the composer.json dependencies looked like this:

    "require": {
        "minishlink/web-push": "5.2.5",
        "ramsey/uuid": "^3.9"
    },
    "require-dev": {
        "mockery/mockery": "1.3",
        "phpunit/phpunit": "5.7.27",
        "wp-coding-standards/wpcs": "^2.3",
        "dealerdirect/phpcodesniffer-composer-installer": "^0.7.0",
        "phpcompatibility/phpcompatibility-wp": "*"
    }

The problem with the upgrade was the initial plan to support PHP 7.1, which according to the WordPress statistics had a 5.9% representation of the total WordPress websites. Some of the dependencies in the latest versions required PHP>=7.2, specially the Push Server library. So, that cut 5.9% potential users:

However, I was able to compensate that 5.9% loss by supporting WordPress starting from 5.2, which initially I was not considering. That was a 6.4% gain:

With that in mind, I submitted the second Release Candidate 1.0.0-rc1 with the suggested changes. You can take a look at all the changes I made here: https://github.com/rwngallego/perfecty-push-wp/commit/2f2322f7848b6cb37465e5776d1f60c981b987d9

In this RC, apart from the minor fixes required after the upgrade, I easily updated the CI pipeline to support the aforementioned PHP/WordPress versions:

    strategy:
      matrix:
        wordpress-version: ['5.2', '5.3', '5.4', '5.5', 'latest']
        php-versions: ['7.2', '7.3', '7.4']

Everything set, all the automated tests green ✅ (the matrix strategy rocks!):

And finally, their approval 🥳:

Publishing

OK, I got access to the Wordpress Plugin directory SVN servers:

Something important here: Once you get the plugin approved, you will need to read carefully all the guidelines related to the publishing, the directory assets (screenshots, logo, banner), the README.txt file and how the whole process works. For more information you can read:

I decided not to publish the plugin right ahead, specially because I noticed some potential improvements (and bugs of course), so I took some days and also prepared the design material. You can see the whole set of changes I made before sending the final version here:

https://github.com/rwngallego/perfecty-push-wp/compare/v1.0.0-rc1...v1.0.1

In this case, I refactored some of the code, improved the usability, added more test scenarios and added the branding stuff. Now the release:

So, right now we have the .zip file and the release published on Github. To publish the plugin in the Wordpress Plugin Directory, you basically need to sync your development work with trunk, change the plugin version, create the tag and then check in the changes:

# change the version in README.txt:
Stable tag: 1.0.0

# change the version in your plugin entry file (`perfecty-push.php`):
 * Version:           1.0.0
 define( 'PERFECTY_PUSH_VERSION', '1.0.0' );

# checkout the upstream
mkdir dist/svn/
svn co https://plugins.svn.wordpress.org/your-plugin-name dist/svn/

# add the changes to trunk
cp -Rp admin includes languages lib public vendor composer.json composer.lock index.php LICENSE.txt perfecty-push.php README.txt uninstall.php dist/svn/trunk

# add the new files if any and check the diff:
cd dist/svn/
svn add --force .
svn stat
svn diff

# if you're happy with the changes, create the tag from trunk:
svn cp trunk tags/1.0.0

# check in your changes:
svn ci -m "Version $SVN_TAG"

Alternatively, I created two additional shell commands to sync and publish the svn package with make svnsync and SVN_TAG=1.0.0 make svnpush, which later I used to setup my deployment pipeline:

create_dist() {
  rm -rf $DIST_PATH
  mkdir -p $SVN_PATH $OUTPUT_PATH
  rm -rf vendor && composer install --quiet --no-dev --optimize-autoloader
  cp index.php vendor/
  cp -Rp admin includes languages lib public vendor composer.json composer.lock index.php LICENSE.txt perfecty-push.php README.txt uninstall.php $OUTPUT_PATH
}

svnsync() {
  create_dist
  svn co -q https://plugins.svn.wordpress.org/perfecty-push-notifications $SVN_PATH
  cp assets/* "$SVN_PATH/assets/"

  # we don't sync vendor if the lock file is the same
  if [[ $(shasum composer.lock | head -c 40) == $(shasum "$OUTPUT_PATH/composer.lock" | head -c 40) ]]; then
    rsync -q -av $OUTPUT_PATH/* $SVN_PATH/trunk --exclude vendor
    echo "## no differences in /vendor, similar lock files ##"
  else
    rsync -q -av $OUTPUT_PATH/* $SVN_PATH/trunk
  fi

  (cd $SVN_PATH && svn add --force . && svn diff && svn stat)
}

svnpush() {
  if (cd $SVN_PATH && svn status | grep -e ^?); then
    echo "There are changes not added to the SVN"
    exit 1
  fi

  if [ -z "$SVN_TAG" ]; then
    echo "You need to provide the tag version as SVN_TAG=1.0.1"
    exit 1
  fi

  if [ -z "$SVN_USERNAME" ]; then
    echo "You need to provide the username as SVN_USERNAME=myname"
    exit 1
  fi

  if [ -z "$SVN_PASSWORD" ]; then
    echo "You need to provide the username as SVN_PASSWORD=mypassword"
    exit 1
  fi

  if [ ! -d "$SVN_PATH/tags" ]; then
    echo "You need to run svnsync first"
    exit 1
  fi

  if [ -d "$SVN_PATH/tags/$SVN_TAG" ]; then
    echo "The tag $SVN_TAG already exists"
    exit 1
  fi

  cd $SVN_PATH && svn cp trunk tags/$SVN_TAG && svn ci -m "Version $SVN_TAG" --username $SVN_USERNAME --password $SVN_PASSWORD

End result, Perfecty Push Notifications published in the WordPress Plugin directory 👏:

https://wordpress.org/plugins/perfecty-push-notifications/

And a deployment pipeline that deploys to Github and Wordpress by just tagging the new version! 🚀

Using the plugin as an end-user

I installed the plugin in one website I have access to, and sent some notifications. In general, it takes about 2 or 3 minutes to send about 2.000 notifications in a basic sever compared to a t2.small. As the plugin uses wp-cron to execute the jobs asynchronously, that timing includes the time gaps between job executions, so it would be less than that.

If you're interested, I will publish another post describing the performance metrics of the plugin, for the moment you can see the impact on the server for one of those jobs:

So, descent results for a very first version! ✨

Take the ride

Wrapping up, if you're still considering writing a plugin, I highly recommend you to take the ride. I created Perfecty Push Notifications as an Open Source alternative for sending Push Notifications from your own Wordpress server for free. I don't know if it will succeed in the Plugin directory or not, however all the things I've learned in the process were valuable, and specially important is the fact that I can share both, the knowledge and the joy of the trip with you 🖖.

Feel free to use it in your websites or recommend it to your WordPress-fans folks. Have a nice one!

Photos

Photo by Bill Jelen on Unsplash

Photo by Roman on Unsplash

Sending a plugin to the Wordpress Plugin Directory

Rowinson Gallego — Tue, 22 Dec 2020 01:58:54 +0000

I'm the creator of the Perfecty Push Notifications plugin. This is a Wordpress Plugin that has a built-in Push API notification integration. You can send push notifications for free, without any third-party dependencies and you retain the data in your server.

Publish time! 🤞

Once I decided it was the right time to publish it in the Wordpress Plugin Directory, I reviewed it again using the detailed plugin guidelines, bundled it and finally uploaded the .zip file to https://wordpress.org/plugins/developers/add/

This was the first step towards publishing the side project I've been recently working on, so I was excited and panicked at the same time. Would it work? What would they complain about the plugin? Would it never take off?

One day later I got an email with a considerable list of recommendations from them:

I will list what those issues were. You can take a quick look at the MR that addresses them here:

https://github.com/rwngallego/perfecty-push-wp/commit/0c9f4e6b7aed12ff0a81b20896f9235663066a4f

Calling files remotely

For the admin area, I'm drawing a simple stat chart using Chart.js.

I initially thought it was not necessary to include it as an enqueued javascript but as a simple direct inline import in the HTML.

<script  
 src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.9.4/Chart.bundle.min.js"
 integrity="sha512-SuxO9djzjML6b9w9/I07IWnLnQhgyYVSpHZx0JV97kGBfTIsUYlWflyuW4ypnvhBrslz1yJ3R+S14fdCWmSmSA=="
 crossorigin="anonymous">
</script>

However, that's not what is suggested in the plugin guidelines, calling third-party systems, so I had to bundle the min.js file with the plugin and mention it in the README.txt:

wp_enqueue_script(
  'chartjs',
  plugin_dir_url( __FILE__ ) . 'js/chart.bundle.min.js',
  array( 'jquery' ),
  $this->version,
  false );

This plugin uses the [Chart.js](https://www.chartjs.org/) library for the admin stats charts.

Data Must be Sanitized, Escaped, and Validated

This is related to this admin page:

I am relying on the well-known WP_List_Table API implementation and honestly didn't catch it, so it was a nice find from them:

perfecty-push-wp/admin/class-perfecty-push-admin-notifications-table.php:118: $ids = is_array( $_REQUEST['id'] ) ? $_REQUEST['id'] : array( $_REQUEST['id'] );
perfecty-push-wp/admin/class-perfecty-push-admin-notifications-table.php:28: 'view' => sprintf( '<a href="?page=%s&action=%s&id=%s">%s</a>', $_REQUEST['page'], 'view', $item['id'], 'View' ),
perfecty-push-wp/admin/class-perfecty-push-admin-notifications-table.php:29: 'delete' => sprintf( '<a href="#" data-page="%s" data-action="%s" data-id="%d" data-nonce="%s">%s</a>', $_REQUEST['page'], 'delete', $item['id'], $action_nonce, 'Delete' ),

perfecty-push-wp/admin/partials/perfecty-push-admin-notifications.php:13: <input type="hidden" name="page" value="<?php echo $_REQUEST['page']; "/>
perfecty-push-wp/admin/partials/perfecty-push-admin-users.php:12: <input type="hidden" name="page" value="<?php echo $_REQUEST['page']; "/>

And it was very easy to fix:

$page = esc_html( sanitize_key( $_REQUEST['page'] ) );

For the issue in the class-perfecty-push-admin-notifications-table.php:118 line, honestly I thought it was sufficient with the intval() filtering to each of the elements:

        $ids = array_map(
            function( $item ) { 
                return intval( $item ); 
            },  
            $ids    
        );

However, I decided to do the recommended sanitization:

        $ids = array_map(
            function( $item ) {
                $item = sanitize_key( $item );
                return intval( $item );
            },
            $ids
        );

Included Unneeded Folders

This was me not knowing how the final structure of the distributable .zip file should look like. I was including some unnecessary folders and files. With their suggestions and reading the guidelines again, I just created a new shell command that copies the required files and the optimized vendor folder:

bundle() {
  CMD=$(plugin_cmd 'rm -rf vendor && composer install --no-dev --optimize-autoloader')
  compose_exec "$CMD"
  cp index.php vendor/
  zip -v -r perfecty-push-wp.zip admin/ assets/ includes/ languages/ lib/ public/ vendor/ composer.json composer.lock index.php LICENSE.txt perfecty-push.php README.txt uninstall.php
}

The final folder structure looks like:

Please use wp_enqueue commands

This is tied to the first issue with Chart.js. As I was including it directly, I was not using wp_enqueue. Now that I'm bundling it, it is already addressed.

And that was it! ✅

Extra

While reviewing their suggestions I noticed I didn't put the dummy index.php file in some of the folders having PHP files so I also did that.

Let's reply to the email

Once I fixed those issues I got back to them and send the link to the distributable .zip file generated from the shell:

~~I'm still waiting for their comments and I expect more input from them, so I'll keep you posted!~~

I've received feedback even before posting this and apparently now I need to address some outdated dependencies from Composer, which ultimately affects the planned minimum PHP version I was intended to support. For that I will create a new post with further details. Thanks for reading!

Photos

Cover: Photo by SpaceX on Unsplash

Why do we close the door?

Rowinson Gallego — Sun, 13 Dec 2020 23:12:38 +0000

Recently, I was reflecting about the way in which I've created my side projects. After many years of try-fail-and-forget, I wondered where those projects ended up. Where were those once upon a time exciting projects? The vast majority on a private repo where dust and glory covered them.

I imagined myself locked in a room, full of enthusiasm and joy building something I think will change the world because what a good idea! However, a new idea or technology appears. It's more exciting, has newer and more powerful features, and maybe, I can focus on that one instead. So, I leave my piece of work in the desk, close the room behind me, and knock on the next door.

I discovered that many of the implementations I intended could have had a better life in other people's hands, instead of just a null instance in life. Maybe there are some golden pieces in there that someone else could take and group them together and continue. Even more, I instantly ruined the opportunity to gain community traction and even support from others, and nobody will never know about what the original idea was.

At the end, I decided not only to not throw the key anymore but to leave the door behind me opened. Why, after investing many hours do we close that door, lock it down, throw the key and knock on the next one? We are closing it after creating fine art, fantastic code that deserves its place in posterity, no matter if it's not perfect, it's our code and we invested time and passion building it. From now on, my side projects will be Open Source.

So, why not leave it open?

Pictures:

Cover image: Photo by Pedro Ramos on Unsplash

[1] Photo by Bhautik Andhariya on Unsplash

[2] Photo by Thom Milkovic on Unsplash

[3] Photo by Murai .hr on Unsplash