DEV Community: Andre Kho

Enable OAuth2 (XOAUTH2) for Sending Emails Using Gmail Account in OJS 3

Andre Kho — Fri, 02 May 2025 05:16:46 +0000

Gmail has been a long-time friend (probably) for developers to help in sending automated emails to users freely right from their systems. But of course the scale of the said systems is small to medium one, estimated up to several tens or hundreds of active users yearly. To be able to use Gmail account in the system, developers simply had to use the email address and password in the mail configuration (using a mailing library) in their projects. But this kind of traditional method has long been criticized because of its unsecure nature, where we had to store the password in separate environment variable or hard-coded it in the code just for the baddie to be see it easily.

In 2024, Google has decided to proceed the transition of authentication method for Gmail accounts from traditional passwords to OAuth2 (details here). This policy has forced many users, especially developers, to refactor their existing systems to be able to use OAuth2 (XOAUTH2) for authenticating Gmail accounts that are being used as system account for mailing purposes. Once a feature called "Less secure apps" access for legacy systems was active in Google Account settings for people to be able to transition to newer method, now it has been effectively removed.

You can read this documentation about how OAuth2 (XOAUTH2) method works.

Open Journal System (OJS) is one of the systems that still use the traditional method for mailing service. Since the change has been implemented throughout the year, people using OJS gradually catch on the issue, which their OJS system can not send automated emails. Have learnt about the Google's change in policy and already mitigate the issue myself before the implementation, I will share what you need to do to update your OJS configuration, so it can be able to send emails with Gmail account once again with OAuth2.

Prerequisites

This workaround is for you if:

Your organization is using Google Workspace. Therefore, the Gmail account which being used for automated mailing purposes is created through Google Workspace.
You have administrative access to Google Cloud resources of your organization (should be automatically associated with the Google Workspace) and credentials of the Gmail account.
You hosted the OJS systems either in Google Cloud's organization (much better) or other platforms.
You are using OJS version 3.
You have sufficient understanding about how PHP project works.

Step 1: Go to Google Cloud Console

Visit Google Cloud Console of your organization. Make sure you are in the Project that contains your hosted OJS if you hosted your OJS with Google Cloud. For you who don't use Google Cloud for hosting, still you can create a Project here first without actually creating VM or any compute solutions.

Step 2: Create OAuth client ID

This steps assumed that you have created an OAuth consent screen. If you haven't, you need to go to the OAuth consent screen first, then fill in information as required. No additional configuration needed there.

Navigate to APIs & Services > Credentials.
Click on + Create credentials button, then click on OAuth client ID.
In the creation form of OAuth client ID page, choose Web application under Application type dropdown. Give name of the client, e.g. "OJS".
Under Authorized redirect URIs section, click on + Add URI button, then type in the full URI of OJS' PHPMailer's OAuth handler script, it should be like: https://[YOUR OJS DOMAIN]/lib/pkp/lib/vendor/phpmailer/phpmailer/get_oauth_token.php
Click Create.
After creation of OAuth client ID, you can always go back to the Credentials page and click on the created OAuth client ID to see Additional information and client secrets, which some of this values will be added to the OJS. Take note of:
- Client ID
- Client secret

Step 3: Modify the `get_oauth_token.php` script

You need to get into the directory of previous URI and edit the get_oauth_token.php directly. Please compare the existing script with the updated version below, before make any changes to yours. Create a backup first!

<?php

/**
 * PHPMailer - PHP email creation and transport class.
 * PHP Version 5.5
 * @package PHPMailer
 * @see https://github.com/PHPMailer/PHPMailer/ The PHPMailer GitHub project
 * @author Marcus Bointon (Synchro/coolbru) <phpmailer@synchromedia.co.uk>
 * @author Jim Jagielski (jimjag) <jimjag@gmail.com>
 * @author Andy Prevost (codeworxtech) <codeworxtech@users.sourceforge.net>
 * @author Brent R. Matzelle (original founder)
 * @copyright 2012 - 2020 Marcus Bointon
 * @copyright 2010 - 2012 Jim Jagielski
 * @copyright 2004 - 2009 Andy Prevost
 * @license http://www.gnu.org/copyleft/lesser.html GNU Lesser General Public License
 * @note This program is distributed in the hope that it will be useful - WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.
 */

/**
 * Get an OAuth2 token from an OAuth2 provider.
 * * Install this script on your server so that it's accessible
 * as [https/http]://<yourdomain>/<folder>/get_oauth_token.php
 * e.g.: http://localhost/phpmailer/get_oauth_token.php
 * * Ensure dependencies are installed with 'composer install'
 * * Set up an app in your Google/Yahoo/Microsoft account
 * * Set the script address as the app's redirect URL
 * If no refresh token is obtained when running this file,
 * revoke access to your app and run the script again.
 */

namespace PHPMailer\PHPMailer;

/**
 * Aliases for League Provider Classes
 * Make sure you have added these to your composer.json and run `composer install`
 * Plenty to choose from here:
 * @see http://oauth2-client.thephpleague.com/providers/thirdparty/
 */
// @see https://github.com/thephpleague/oauth2-google
use League\OAuth2\Client\Provider\Google;
// @see https://packagist.org/packages/hayageek/oauth2-yahoo
use Hayageek\OAuth2\Client\Provider\Yahoo;
// @see https://github.com/stevenmaguire/oauth2-microsoft
use Stevenmaguire\OAuth2\Client\Provider\Microsoft;

if (!isset($_GET['code']) && !isset($_GET['provider'])) {
    ?>
<html>
<body>Select Provider:<br/>
<a href='?provider=Google'>Google</a><br/>
<a href='?provider=Yahoo'>Yahoo</a><br/>
<a href='?provider=Microsoft'>Microsoft/Outlook/Hotmail/Live/Office365</a><br/>
</body>
</html>
    <?php
    exit;
}

// require 'vendor/autoload.php';
// require($_SERVER['DOCUMENT_ROOT'] . '/lib/pkp/lib/vendor/autoload.php');
// Below is the new location of autoload
require('/var/www/lib/vendor/autoload.php');

session_start();

$providerName = '';

if (array_key_exists('provider', $_GET)) {
    $providerName = $_GET['provider'];
    $_SESSION['provider'] = $providerName;
} elseif (array_key_exists('provider', $_SESSION)) {
    $providerName = $_SESSION['provider'];
}
if (!in_array($providerName, ['Google', 'Microsoft', 'Yahoo'])) {
    exit('Only Google, Microsoft and Yahoo OAuth2 providers are currently supported in this script.');
}

//These details are obtained by setting up an app in the Google developer console,
//or whichever provider you're using.
$clientId = '[YOUR CLIENT ID HERE]';
$clientSecret = '[YOUR CLIENT SECRET HERE]';

//If this automatic URL doesn't work, set it yourself manually to the URL of this script
//$redirectUri = (isset($_SERVER['HTTPS']) ? 'https://' : 'http://') . $_SERVER['HTTP_HOST'] . $_SERVER['PHP_SELF'];
$redirectUri = 'https://[YOUR OJS DOMAIN]/lib/pkp/lib/vendor/phpmailer/phpmailer/get_oauth_token.php';

$params = [
    'clientId' => $clientId,
    'clientSecret' => $clientSecret,
    'redirectUri' => $redirectUri,
    'accessType' => 'offline'
];

$options = [];
$provider = null;

switch ($providerName) {
    case 'Google':
        $provider = new Google($params);
        $options = [
            'scope' => [
                'https://mail.google.com/'
            ]
        ];
        break;
    case 'Yahoo':
        $provider = new Yahoo($params);
        break;
    case 'Microsoft':
        $provider = new Microsoft($params);
        $options = [
            'scope' => [
                'wl.imap',
                'wl.offline_access'
            ]
        ];
        break;
}

if (null === $provider) {
    exit('Provider missing');
}

if (!isset($_GET['code'])) {
    // If we don't have an authorization code then get one
    $authUrl = $provider->getAuthorizationUrl($options);
    $_SESSION['oauth2state'] = $provider->getState();
    header('Location: ' . $authUrl);
    exit;
// Check given state against previously stored one to mitigate CSRF attack
} elseif (empty($_GET['state']) || ($_GET['state'] !== $_SESSION['oauth2state'])) {
    unset($_SESSION['oauth2state']);
    unset($_SESSION['provider']);
    exit('Invalid state');
} else {
    unset($_SESSION['provider']);
    // Try to get an access token (using the authorization code grant)
    $token = $provider->getAccessToken(
        'authorization_code',
        [
            'code' => $_GET['code']
        ]
    );
    // Use this to interact with an API on the users behalf
    // Use this to get a new access token if the old one expires
    echo 'Refresh Token: ', $token->getRefreshToken();
}

Step 4: Replace `vendor` packages manually

Due to the obsolete packages being used by this particular script, you actually need to update the dependencies of this PHPMailer package. However, it may be complicated since it's also a dependency for OJS. For this workaround, you can use my vendor packages from here. Extract this archive into a separate directory aside of your OJS in the server, e.g. /var/www/lib (as I pointed out in the PHP script above).

If you want to get yourself the new vendor folder, you may set up a local PHP project (just create an empty folder anywhere you want). Inside this folder, run below command:

composer require league/oauth2-google

After adding the dependencies, you can archive the vendor directory in the local project and upload it to your OJS server. Then, extract the archive into separate folder, accessible by the OJS like above mentioned, e.g. /var/www/lib.

Step 5: Update `Mail.inc.php` dependency

Put this require line at the first lines of the Mail.inc.php script located in lib/pkp/classes/mail, right before the actual code.

require('/var/www/lib/vendor/autoload.php');

This line help to make sure the Mail class can load dependencies right to the PHPMailer package.

Step 6: Retrieve a Refresh Token

After setting things up, you need to visit the URI of get_oauth_token.php, so it should be: https://[YOUR OJS DOMAIN]/lib/pkp/lib/vendor/phpmailer/phpmailer/get_oauth_token.php. In this page, click Google link and you may be prompted with Google Login page. Login with your purposed Gmail account.

You can see a line of refresh token after a successful login. Copy this value and paste somewhere else. We need this refresh token for new configuration of OJS.

Step 7: Add XOAUTH2 configuration to `config.inc.php`

Edit the config.inc.php in the root directory of your OJS, focus on Email settings section here. Adjust the configuration like this example:

[email]

; Use SMTP for sending mail instead of mail()
smtp = On

; SMTP server settings
smtp_server = smtp.gmail.com
smtp_port = 587

; Enable SMTP authentication
; Supported mechanisms: ssl, tls
smtp_auth = tls
smtp_username = [YOUR GMAIL ADDRESS HERE]
smtp_password = [YOUR GMAIL PASSWORD HERE]

smtp_authtype = XOAUTH2
smtp_oauth_provider = Google
smtp_oauth_email = [YOUR GMAIL ADDRESS HERE]
smtp_oauth_clientid = [YOUR OAUTH2 CLIENT ID HERE]
smtp_oauth_clientsecret = [YOUR OAUTH2 CLIENT SECRET HERE]
smtp_oauth_refreshtoken = '[YOUR OAUTH2 REFRESH TOKEN HERE]'

; Allow envelope sender to be specified
; (may not be possible with some server configurations)
allow_envelope_sender = On

; Default envelope sender to use if none is specified elsewhere
default_envelope_sender = [YOUR GMAIL ADDRESS HERE]

; Force the default envelope sender (if present)
; This is useful if setting up a site-wide no-reply address
; The reply-to field will be set with the reply-to or from address.
force_default_envelope_sender = On

Step 8: Test the mailing feature

Finally, you can test the changes you have made so far by creating a dummy submission in your OJS website, or try anything else that can trigger the outgoing mail action from the site.

Conclusion

OJS has been used by many institutions for years to help managing scienfitic articles and publications. The dynamics from Google especially in the rise of a more secure method to authenticate email addresses using OAuth2 make legacy systems like OJS has to change, where email notifications are one of the most important feature for users may be disrupted if they are using Gmail. While there's maybe difficulties faced by institutions to update their OJS systems following the official documentation, when they are using Gmail accounts to send emails, we can still use some workaround here. Not the best one, but I hope it can help you get there.

Easy Uptime Monitoring Tool with Uptime Kuma in Google Cloud Platform

Andre Kho — Wed, 08 Dec 2021 09:05:38 +0000

Monitoring our things like websites and APIs eventually become one of major task right after we have successfully deployed them. We want to make sure that our services can be online and healthy all the time for users. To get this done, we usually don't want to waste time to develop our own monitoring tool just for ourselves.

Instead, we can use already established online service like UptimeRobot. We can simply register to them, put our URLs that need to be monitored, then we can see how our services performed through provided charts. But just like any third party services, there are always limitations for every free tiers. If we need to have more advanced features, then we pay for paid tier which is kinda waste of money depending of what we actually need. For example, we simply want the uptime checking cycle reduced to some minutes less than five minutes of free tier.

Hence, there is Uptime Kuma. This is the best open-source monitoring tool yet I've ever found. It's free to use, inspired by UptimeRobot (much better and simpler), no tiered restrictions and best of all, it's so easy to deploy for ourselves! You can go check their official Github repository here.

In this post, anyway, I'm about to guide you to how we can easily deploy Uptime Kuma to Google Cloud Platform. It's pretty straightforward to what I'm going to demonstrate, if:

We just need to deploy the tool without using any domain/subdomain
We want this tool accessible through internet
We want cheap deployment (yes, I'm not gonna put 'cheapest' since there will always the free way)

Now let's get started!

Notes: I'm going to use Compute Engine, instead of App Engine Flexible, Cloud Run or any other services that can host a Docker container. For what I have experienced so far, this is the easiest way to go.

Create a Compute Engine instance with Container Deployment

Uptime Kuma is stateful. It stores some persistent data in itself using local SQLite (based on the repo). Hence, they give ways for us to deploy the tool, which one is by deploying Docker container. The official Docker image is ready to deploy, no need for us to install and configure the environments.
However, one step we still need to do before deploying the container is to create a persistent volume. Of course, this is well supported by Compute Engine, but there are something we need to configure.

Now let's create our instance through this button in Compute Engine page.

Then, we configure our instance like these:

Give name to our instance and choose location that covered by GCP's Free Tier.
Choose machine type that covered by GCP's Free Tier as well.
Next, we configure our boot disk type. Click on CHANGE button. On the form, choose Standard persistent disk, then click SELECT button. (This disk type is covered by GCP's Free Tier)

Scroll down a little bit, you will find this section. Click DEPLOY CONTAINER button.
In the shown form, first fill in these fields. We type in the URL of Uptime Kuma's latest image from Docker Hub: registry.hub.docker.com/louislam/uptime-kuma:1. Then if you need it, you can check the options here.
Scroll down, you'll find this section. Click on ADD VOLUME.

Now before we continue, you can choose one of two options below to create a Volume for our container. Option #1 is using TmpFS volume type, Option #2 is using Disk volume type.

Option #1, using TmpFS volume type

Choose TmpFS volume type. Even it is a temporary file storage, based on Compute Engine documentation here, it states that:

with tmpfs on a Compute Engine container, the volume and its data persist across container restarts and are deleted only on VM restart.

Click DONE and finally SELECT button. Skip Option #2 steps below.

Option #2, using Disk as volume type

Close the Container deployment form for now. Then, scroll down until you find this advanced section. Click on it and scroll to Disks section.
Click on ADD NEW DISK to open disk creation form. Fill in configuration like below. I simply choose zonal, blank, standard persistent disk with 10 GB in capacity. Click SAVE button when finished.
Now back to steps in Container Deployment form, in Volume mount section, choose Disk as volume type. Then pick your just created Disk and fill in other fields like below. Click DONE and finally SELECT button. Now we can move on to next steps.

Further, we have to give our instance a network tag, for example: allow-kuma. Soon after we create the instance, we need to configure firewall to this instance. You may also want to configure a static external IP address with Standard network tier in this section.
Finally, click on CREATE button at the bottom of the screen.

Configure Firewall rule

Now we already have our instance up and running, but we can't access it through its external IP address yet. We need to open port number 3001 to this instance, hence the given network tag.

Go to Firewall page.
Now set these settings. Click CREATE when you done.
- Name: allow-kuma
- Network: choose the VPC where the Compute Engine instance exists
- Target tags: allow-kuma
- Source IPv4 ranges: 0.0.0.0/0
- Protocols and ports: Check on Specified protocols and ports, then check tcp and type in number 3001.

Just like that, our Uptime Kuma already up and ready to use. Open it through the VM's external IP address and concate with port 3001 like so: http://123.123.13.12:3001.

Now, let's see our cost estimation for those configuration. We assume these variables:

We are using Standard networking tier
We are using Compute Engine instance that covered by GCP's Free Tier.
Monthly network egress comes from our instance based on per minute basis pings for 7 monitors is roughly around 0.5 GiB.

You can refer to this Google Cloud pricing calculator estimate link

We are only incurred cost around USD 0.44 per month for a rich features monitoring tool! Now that's cheap 💰💰

That's it for this post. Further you can explore Uptime Kuma's features and integrate it with social app like Discord. Also, it provides a customizable status page. They are pretty neat and helpful. Save and share this post if you like!

Deploy Laravel 8 to Compute Engine instance (LEMP stack)

Andre Kho — Fri, 03 Dec 2021 05:33:25 +0000

In this post I'll walk you through to get a Laravel 8 app running on a fresh Compute Engine instance in NGINX way. We'll be using these assumptions:

Already have a Laravel 8 project stored in a remote repository (I'm using Github for this post).
Stateful solution which means the database and application code are placed together in the instance.
You want to use Google Cloud Platform services to host your application.
You want to have full control over server environments eg. web server configurations.
You want your app to just works as is.
You have familiarity with Linux commands.
You need optimal configuration for your app.
You may skip the database steps if you are not actually use it on your project.

Now, let's get started.

Step 1: Create a Compute Engine instance

First thing to do is to create and configure our VM instance to serve our app. In Cloud Console, refer to Compute Engine menu then click on CREATE INSTANCE button as shown below.

Then in create instance form, fill and set properties for our instance. My configuration goes like these:

Choose your instance name wisely as it can't be changed later after creation.
Pick your nearest region and zone for lower latency.
Choose lowest possible machine class and type to avoid huge charge to your bill, even if it's just for trial use.

On boot disk configuration, choose your most familiar operating system. If it's required, you may choose latest LTS version.
Choose SSD persistent disk and smallest size as starting point (10 GB is the lowest). SSD comes with greater performance compared to others except the Extreme one, but costs higher.

Enable all HTTP and HTTPS access, just in case you need to configure SSL certificate for your instance.

On advanced configuration, the Network section, choose your existing VPC network. (I already have a custom one called local-vpc. You may already have the default one.)
Set a static external IP address as shown below.

Set a static external IP address now so you won't bother to reconfigure DNS A record to this instance everytime its IP address changes.

On advanced configuration, the Security section, enable Shielded VM for increased security to your instance.
You may want to add a Public SSH key in this section, so you can access your instance securely from your workstation or PC.

Click on CREATE button to create the instance.

Step 2: Install and configure server environments

Now your instance already up and running. We need to install our necessary environments for our app.

Click on SSH button (or run ssh command from your PC that access this VM's external IP address)

Now first in your SSH terminal, run these commands:

Set your timezone

sudo timedatectl set-timezone Asia/Jakarta

Check for any updates

sudo apt-get update

Do upgrade according to updates

sudo apt-get upgrade

(Optional) Do reboot the instance

sudo reboot

At this point, access our instance once again and we will install our environments:

Install NGINX

sudo apt install nginx

Install MariaDB server. At this point, follow the instruction to make our MariaDB server secure. You'll do the following:
- Authenticate as root user (just press enter to move forward)
- Set new root password (remember the password you entered here!)
- Accept everything (don't forget to read the instruction, though)

sudo apt install mariadb-server
sudo mysql_secure_installation

Create a new database and database user for our app

sudo mysql

mysql> CREATE DATABASE 'mydb';
mysql> CREATE USER 'mymy'@'localhost' IDENTIFIED BY 'myPassw0rd';
mysql> GRANT ALL ON 'mydb'.* TO 'mymy'@'localhost';
mysql> FLUSH PRIVILEGES;
mysql> EXIT

Install unzip so our Composer can do its work when installing dependencies

sudo apt install unzip

Add PPA repository for NGINX and PHP

sudo add-apt-repository ppa:ondrej/php && sudo apt update
sudo add-apt-repository ppa:ondrej/nginx && sudo apt update
sudo apt upgrade

Install PHP-FPM and some 'necessary' PHP modules (refer to Laravel official docs anyway)

sudo apt install -y php7.4-fpm php7.4-curl php7.4-gd php7.4-json php7.4-mbstring php7.4-mysql php7.4-opcache php7.4-xml php7.4-xmlrpc php7.4-fileinfo php7.4-imagick php-pear
sudo apt upgrade

Check PHP CLI version with php -v command. If the PHP CLI version check returns other than version 7.4, run below command to change its version. Type a number option that shows version 7.4 then press Enter.

sudo update-alternatives --config php

Create an NGINX config file under /etc/nginx/sites-available/ directory. Name the config file either with a domain name you have or just give it main or anything you want. I'll go with example.com.

sudo nano /etc/nginx/sites-available/example.com

This will open a NANO text editor ready to create the file. Copy and paste the code below to the editor.

server {
  listen 80;

  # Use your domain/subdomain name here
  server_name example.com;

  # We will clone our "repo-name" repository which
  # contains our Laravel project to /var/www/ directory later.
  # Change this path according to your repository name
  # and end with /public.
  root /var/www/repo-name/public;

  index index.php;

  add_header X-Frame-Options "SAMEORIGIN";
  add_header X-Content-Type-Options "nosniff";

  charset utf-8;

  location / {
    try_files $uri $uri/ /index.php?$query_string;
  }

  location = /favicon.ico { access_log off; log_not_found off; }
  location = /robots.txt  { access_log off; log_not_found off; }

  error_page 404 /index.php;

  location ~ \.php$ {
    fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
    fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
    include fastcgi_params;
  }

  location ~ /\.(?!well-known).* {
    deny all;
  }

  # The access log file name is up to you.
  access_log /var/log/nginx/access_laravel.log;

  # And this error log file name as well is up to you.
  error_log /var/log/nginx/error_laravel.log;
}

Make sure everything is correct, then press Ctrl + X, press Y, and finally press Enter. This will save the new file and exit the editor.

Create a symbolic link to the config file in /etc/nginx/sites-enabled/ directory, so NGINX can read our file from there.

sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/

Unlink existing default configuration

sudo unlink /etc/nginx/sites-enabled/default

Test our NGINX configuration

sudo nginx -t

If no errors come up, restart the NGINX service to load our new configuration

sudo service nginx reload

Now, we will install additional environments for our Laravel app.

Install Node Version Manager (NVM) to run npm installation in the project later.

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
source ~/.bashrc

Install NodeJS latest LTS version. Use nvm list-remote to list Node versions (current latest LTS version is Gallium)

nvm install lts/gallium

You may want to check the installed version of Node and NPM by running these commands.

node -v
npm -v

Install Composer (Required)

curl -sS https://getcomposer.org/installer -o composer-setup.php

HASH=`curl -sS https://composer.github.io/installer.sig`
php -r "if (hash_file('SHA384', 'composer-setup.php') === '$HASH') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"

sudo php composer-setup.php --install-dir=/usr/local/bin --filename=composer

Step 3: Pull your app from a remote repository

Now it's time to put our app to this VM. I'll go with more secure way to clone our repository by using SSH keys. You may use your existing SSH keys or generate a new one.

First, we need to generate a new SSH key pair. Please refer to this site to learn about how to use SSH.

ssh-keygen -t rsa -b 4096 -C "<your-Github-email>"

Make sure change "your-Github-email" to your actual email address used in Github (it's not mandatory, to be honest. You can fill anything, but email is recommended as it's easier to identify).
You'll be asked file name for the key. You may just press Enter to use default file name.
Then, you'll be asked to type in a passphrase. It's a best practice to type in the passphrase, but for our case, it's not necessary to do so. Just press Enter twice so you won't use any passphrase.
Finally, the SSH key pair is generated and stored within .ssh folder in current user directory (~/.ssh/). There will be two files: one with .pub extension is called public key, the other one is called private key.

Next step, we will put our public key content to our Github account (if you are using other service that Github, refer to their documentation about how to clone a repo using SSH).
Copy our public key content which usually begins with ssh-rsa xxxx.
Then, go to your Github account settings page > SSH and GPG keys. Click on New SSH key button. Type a name for the key (anything but noticeable) and paste the public key in provided text area. Click on Add new key button.

Now, we can start cloning our repo from Github securely. Here are the steps.

(Required) Secure our SSH private key by changing file permission to read-only by its owner (current user who creates the key pair.

chmod 400 .ssh/id_rsa

Activate ssh-agent so we can start use SSH.

eval `ssh-agent -s`

Add our private key to the SSH agent. Remember your key file name.

ssh-add id_rsa

Test whether we can connect to Github via SSH. Type yes and press Enter to confirm connection when being asked.

ssh -T git@github.com

If it returns a message like: "Hi, <user>! You've successfully authenticated, but GitHub does not provide shell access.", then it's successful connection! We can continue to clone our repo to /var/www directory by running this command (change to your username and repository name accordingly).

cd /var/www/
sudo chown $USER:$USER .
git clone git@github.com:<your-Github-username>/<your-repo-name>
sudo chown root:root .

What I'm doing upthere is changing ownership of /var/www folder to current user, so we can run git clone command as current user. Thus, our cloned repo will be owned by it. Then, we return the ownership of /var/www directory back to root user (not everything inside it). Pay attention to cd command.

Step 4: Install Laravel dependencies and configure project environment variables

At this point, we already have our cloned project inside /var/www directory. Make sure you are inside the project directory by using cd command.

First thing is to create a .env file by copying from .env.example file.

cp .env.example .env

Then, open and edit the copied .env file with NANO editor.

nano .env

Change respective environment variable values to suit your requirements. For my case, I'll go with updating these values.

APP_ENV=production
APP_DEBUG=false
DB_DATABASE=my-db
DB_USERNAME=mymy
DB_PASSWORD=myPassw0rd

Remember your database user, its password, and the database whom its database user can access (refer to Step 2). Press Ctrl + X, then Y, and finally press Enter to save.

We will install all Laravel modules and its dependencies by running these commands.

composer install --optimize-autoloader --no-dev
npm install
npm run dev

Moving forward, we need to generate application key which is mandatory.

php artisan key:generate

Next we will create a symbolic link for our /storage folder.

php artisan storage:link

One more step, we can migrate our database. You may skip this if you are actually not using database.

php artisan migrate:refresh --seed

Finally, we need to change ownership for www-data user (web server user) so it can access and serve our app. Change repo-name to your actual repository or folder name.

sudo chown -R www-data:www-data /var/www/repo-name

There are many alternatives to this. One way, you can add www-data user to current user group and vice versa.

At last, we can access our website through external IP address of our VM. There are more steps if we want to make use of our own domain instead of server's IP address, but those are for future post.

Conclusion

It's pretty hefty to just deploying one website, especially if there are many requirements behind the scene. As the title said, one way we can deploy our Laravel app is just by leveraging most used server engine like NGINX, then we can put every other Laravel dependencies in place inside the server in a pretty straightforward way (read: running through commands).
One thing to note that we need to pay attention about security of the server itself, which a small part of it is by making use of SSH. We can further improve this by configuring correct firewall rules, install SSL certificates, reconfigure SSH access, optimize NGINX configurations, etc. If you mind though, you can configure high availability to your app even with Compute Engine by making our VMs stateless.

That's all for this post. Hope it helps you. Don't hesitate to ask me if you are unsure or you want add something that I might be missing in this post. Thank you for your time reading this 🤗

Guide to Setup a Moodle website in Google Cloud Platform (GCP) with HTTPS

Andre Kho — Fri, 25 Jun 2021 18:14:14 +0000

Moodle has been great for supporting many schools, campuses, and even companies in digitalized education system. Moodle is quite stable to be deployed by IT teams or individual for their organizations, simple to customized. It can be deployed offline or online depending on needs and situation. There are many third-party software installers that can ship and deploy Moodle almost instantly, most of them are managed by another third-party hosting services.

While it's pretty quick and simple to start, but at some point it's becoming difficult to maintain and keep up-to-date. We may not have a full control of the server that hosts our Moodle, hence we can't setup some new requirements needed or do advanced tinkering. We may overprovision resources that host the Moodle application, when most of the time it's always in underutilization. We also may have corrupted Moodle installation from those third-party tools, that we don't notice at first but then it's showing everywhere. We also may forget at some point to check whether our website is using a secure protocol (HTTPS) or not. Yeah, lot of risks, but it's not the end of the world.

In this post, I will guide you how to setup a Moodle LMS website hosted in a Compute Engine (CE) instance in Google Cloud Platform (GCP). The application will be secured using free and open source SSL certificate service so we don't have to pay if we don't want to.

Post Updates

30 June 2021: Modify cron steps as we only setup using PHP 7.4 and add opinion about scaling Moodle
29 June 2021: Add more reference and set server time zone

Assumptions 🤔

Below is the list of environment and app version used in this guide (at the time this post out):

Moodle 3.11+
PHP 7.4.20 and 8.0
MySQL 8
Ubuntu 20.04 LTS

While it's based on my experience, please note that you may adjust some steps to suit your needs. Another disclaimer is that the steps I'm going to show can be subjective, so I make some assumptions:

You already have an active Billing Account set for your organization (or your personal Google Account).
You are already prepared to pay for using GCP resources and you aware that pay-per-use model needs extensive monitoring of resource usage to avoid overpricing.
You already have a DNS Records management set elsewhere outside GCP (eg. cPanel user account).
You already have control of a school/campus domain name.
You are trying to setup your Moodle LMS to be accessible on a subdomain or domain.
Your user size won't be too big to handle, hopefully (approx. 10-100 concurrent users).
You already have familiar knowledge of GCP, their policies, and techniques.

Overview 📑

There are 5 main steps to setup everything, listed as below:

Provision a Compute Engine instance
Install server environments
Make our subdomain/domain secure
Get Moodle through Git
Setup Moodle specific configurations

Step 1: Provision a Compute Engine instance

First things first, you have to create a CE instance in GCP. It's pretty easy and straightforward. You may want to use gcloud shell commands, but I prefer to use the Cloud Console web interface.

As for initial setup, you can start with as low as below specifications:

Region/Zone: choose the nearest possible one from your place (the price may be higher, but it provides lower latency). Our instance is zonal resource.
Machine configuration:
- Machine family: General Purpose
- Series: E2
- Machine type: e2-micro (1 shared-core up to 2 vCPUs and 1 GB memory which only costs around US$11.14 per month on full run)
Boot disk:
- Operating system: Ubuntu
- Version: latest LTS possible (choose 20.04 LTS as of the time I wrote this post)
- Boot disk type: SSD persistent disk
- Size: 10 GB (only costs around US$2.21 per month)
Firewall: Check both Allow HTTP traffic and Allow HTTPS traffic
Under Networking tab in advance configuration, click on one of the Network interfaces (should be default at first). You may want to use Static External IP instead of Ephemeral.
(Optional) SSH keys: Provide your public SSH key(s) generated from your PC. Make sure your private SSH key(s) is secured in your user directory. Although it's optional, but I use this to provide access to the VM directly from my Terminal without bother opening Google Cloud Console in browser.

Noted that this specification is the lowest possible one and hopefully cost-effective. You may choose higher resources, but it's a good thing to start small when you are working in the cloud.

Step 2: Install server environments

Please make note of your instance's external IP address.

If you choose to provide SSH key(s) to the instance, then you can open your terminal and try to connect through it with this command (I'm using Windows Terminal btw):

ssh <your_username>@<external_ip> -i <full/path/to/your/private/keys>

Otherwise, you can click on SSH button inline with instance name in Compute Engine > VM Instances table. This will open a new browser window that showing terminal access to the instance.

Set server time zone

Before we continue further, we need to make sure that our server time zone matches with ours. We can check via below command.

timedatectl status

If it shows different time zone than yours, then you need to change it. We can check available time zones with below command.

timedatectl list-timezones

Find your time zone (mine is Asia/Jakarta, which is UTC+7), and then execute command below to set your time zone.

sudo timedatectl set-timezone <your time zone>

At this point, I reboot the machine, although the server's time zone already changed. Just to make sure, of course.

Environment preparations

Now we are going to install environments needed, based on the official guide for installing Moodle in Ubuntu (check it here). I will show you here anyway.

Update Ubuntu components

sudo apt-get update
sudo apt-get upgrade

Add apt repository to track updates for PHP

sudo add-apt-repository ppa:ondrej/php
sudo apt-get update

Install Apache and MySQL

sudo apt install apache2 mysql-client mysql-server php7.4 libapache2-mod-php7.4

Run MySQL secure installation. You'll be asked for a new ROOT password using options of validation (choose 1 = MEDIUM). Then you'll just confirm following questions with Y (yes). This has to be done to make your MySQL server as secure as possible by disabling remote root access and provide strict validation when creating passwords.

sudo mysql_secure_installation

Install additional software or dependencies. Please be aware that we are going to install components for PHP 7.4.

sudo apt install graphviz aspell ghostscript php7.4-pspell php7.4-curl php7.4-gd php7.4-intl php7.4-mysql php7.4-xml php7.4-xmlrpc php7.4-ldap php7.4-zip php7.4-soap php7.4-mbstring

Restart Apache server to make sure all components are loaded successfully.

sudo service apache2 restart

Install Git (for Step 4).

sudo apt install git

Check listening ports

We need to check whether port 80 (not secure) and 443 (secure) is listening from your machine by running this command.

sudo ss -tulwn

If port 443 is not listed, we need to install ssl module for Apache.

sudo a2enmod ssl

Restart Apache once again

sudo service apache2 restart

Now check again all listening ports. Port 443 should be listed as LISTEN.

sudo ss -tulwn

Step 3: Make our subdomain/domain secure

Now we need to make sure that our subdomain (or main domain) uses HTTPS. That means, we need an SSL certificate. We will use the free way to obtain that.

First, in your DNS records manager, add or modify two A records just like below. By doing this, we are going to make sure that our subdomain/domain will pinpoint to your machine external IP address. In order (Name, TTL, Type, Record):

example.com, 14400, A, (your external IP)
www.example.com., 14400, A, (your external IP)

Then, we are going to install Certbot, a free, open source software tool for automatically using Let’s Encrypt certificates on manually-administrated websites to enable HTTPS (check it here).

You can follow the specific instructions through this link for our case (Ubuntu 20.04 and Apache).

If you follow all the Certbot instructions and get a successful result, then you should be able to access your subdomain/domain using HTTPS. Now that's easy and under your control.

Step 4: Get Moodle through Git

Git is what is called a "version control system". By using git, it will much easier down the road to update the Moodle core application. We will use /opt directory for this installation.

cd /opt

Download the Moodle code and index.

sudo git clone git://git.moodle.org/moodle.git

⚠ Warning: Because we are using e2-micro (as I was), this command will somehow make our machine disk write and read to be at peak for several minutes (if not hours) and your SSH access might be interrupted because of timeout. Take your time, sip a cup of tea ☕. It will be finished anytime soon.

Then, change directory into the downloaded Moodle folder.

cd moodle

Retrieve a list of each branch available. Press Q to quit seeing.

sudo git branch -a

Corresponding with Moodle convention on branch naming, we will use the latest Moodle version (as the time of writing, it's Moodle 3.11). Tell git which branch to track or use.

sudo git branch --track MOODLE_311_STABLE origin/MOODLE_311_STABLE

Check out the Moodle version specified.

sudo git checkout MOODLE_311_STABLE

Step 5: Setup Moodle specific configurations

Copy installation to web root

Now we have our Moodle installation, we need to copy that to the webroot (it's usually under /var/www/html). Then we need to create moodledata directory to store Moodle-generated files. We should change both directories' permissions accordingly.

sudo cp -R /opt/moodle /var/www/html/
sudo mkdir /var/moodledata
sudo chown -R www-data /var/moodledata
sudo chmod -R 777 /var/moodledata
sudo chmod -R 0755 /var/www/html/moodle

Change web root entrypoint

At this point, we should have PHP 7.4 and MySQL 8 running. And also, our Moodle application is now accessible through https://(your domain)/moodle. This is quite annoying. We need to change the web root config so that our user can access through https://(your domain).

sudo nano /etc/apache2/sites-available/000-default.conf

Now change in the line of DocumentRoot /var/www/html to DocumentRoot /var/www/html/moodle. Save the file by presing Ctrl+X > confirm with Y > Enter to save with default file name (we are using Nano text editor).

Restart the Apache server.

sudo service apache2 restart

Create new database and user

We need to create a user for our Moodle database. Open MySQL as root, use your MySQL root password from Step 2.

sudo mysql -u root -p

Now we are in MySQL shell.
Create database.

mysql> CREATE DATABASE moodle DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

Create user. Use your username and password of your choice.

mysql> CREATE USER 'moodledude'@'localhost' IDENTIFIED BY 'passwordformoodledude';

Grant some privileges to the user.

mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO 'moodledude'@'localhost';

Quit the shell.

mysql> quit;

Update opcache and some PHP configuration

Before we continue, we should update some opcache and PHP configuration according to Moodle docs. Please review the required settings for:

For opcache config file, you can open from this location (use ls to check opcache.ini name file:

cd /etc/php/7.4/apache2/conf.d/
ls
sudo nano 10-opcache.ini

For PHP config file, you can open from this location:

cd /etc/php/7.4/apache2/
ls
sudo nano php.ini

Please update all required settings before we continue. Note that from my case, I don't need to uncomment extension sections in php.ini. They are already enabled since component installation at Step 2.

After you finished, restart the Apache server to load the new settings.

sudo service apache2 restart

Continue installation from web interface

Now we can continue our Moodle installation from its web interface. Before that, change the permissions on /var/www/html/moodle directory so the installation can succeed smoothly.

sudo chmod -R 777 /var/www/html/moodle

Open your browser and access https://(your domain) and follow the prompts.

Change path of moodledata to: /var/moodledata
Change database type to: mysqli
Change database settings:
- Host server: localhost
- Database: moodle
- User: moodledude (the user you created when setting up the database)
- Password: passwordformoodledude (the password for the user you created)
- Tables Prefix: mdl_
Environment checks. At this point every thing should be in green OK. If there are any warnings, let me know.
Next and next and confirm installations.

At this point, installation should finished successfully (just wait). Follow next prompts for creating Admin user and website configuration.

Change system path

Now Moodle is already up and running. We need to update some little configuration a little more.
Navigate to Site Administration > Server > System Paths. Input path settings like the following:

Path to du: /usr/bin/du
Path to aspell: /usr/bin/aspell
Path to dot: /usr/bin/dot

Save your changes.

Setup cron job for Moodle

As required by Moodle, we need to setup a cron job (as mentioned in this documentation). We can check the PHP CLI configuration using below command.

php -i | grep php.ini

Make sure that the PHP version used by CLI is the same with our initial installation, which is 7.4.

Now we add our cron job by using crontab.

crontab -u www-data -e

At first, you'll be asked which editor to use. Choose nano. Then, add the following at the end of the opened file.

* * * * * /usr/bin/php  /var/www/html/moodle/admin/cli/cron.php >/dev/null

Save your changes. Now our Moodle has its cron job running every 1 minute.

Don't forget to revert Moodle directory permissions

Run this command to revert /var/www/html/moodle permissions.

sudo chmod -R 0755 /var/www/html/moodle

Monitor the usage

Don't forget to monitor the instance usage and cost. Simple but important.

Known Issues 🤯

If you try to access VM directories via Visual Studio Code, you may encounter lag and sometimes timeout. In my case, the disk IOPS and I/O usage is full for several minutes and hours when the issue happens. Do you have any ideas on this? Solution: We have to scale up our machine type.
Certbot may fail to give you SSL certificate if you don't properly setup DNS records and open port 443.
Certbot may fail to renew SSL certificate on first check, even we already have a proper setup. Try executing the certbot command again.

Next steps ⏩

Below is the list of what you can do further:

Start installing plugins that suit your needs.
Scale up your instance by turning it off first then change its machine type and disk size. Restart it whenever you are ready.
Setup specific Firewall policies for your subnetworks.
Avoid directory ownership mismanagement. You don't want to accidentally allow unintended access and authorization that can alter your Moodle source.
Use external database authentication and enrolment plugins to speed up administration routines. I will cover this in the upcoming post.
If you follow Moodle installation steps from official docs, pay attention on clamav component. This antivirus will cause hourly huge spikes in CPU utilization to process virus database. You may want to scale up your instance if you want to make use of it.

Opinion 💭

There is "official" Docker image of Moodle, but it's solely used for development purposes, not production.
You may think of using Kubernetes architecture to get autohealing and autoscaling features, but that may bring a lot of technical difficulties to setup. Always start simple.
As stated by Moodle docs, Moodle LMS itself can be clustered (or at least scaled out). However, it's not necessary and will require complex architecture work to do so. Instead, Moodle LMS can be scaled up by just increasing memory, CPU, and disk. In this way, of course, Moodle is at higher risk of downtime since our single VM only exists in a single zone in a region.
Actually, there is a way to separate Moodle source app, moodledata, and the database so it can be scaled out. But that will incur higher cost than we expected. For example, GCP itself allows use of Cloud Storage to be mounted for VMs using FUSE. The idea is to use it as a place for moodledata. But we will be charged with Cloud Storage pricing plus its performance may not significantly sufficient for Moodle to store and process its data (lower performance than SSD and limited concurency capability).

Conclusion 🏁

That is a hell lot of steps to install Moodle in GCP. But, we already established our very own Moodle installation. We have full control over resources and that is the main point and a good thing! We only pay for what we use (or provision to be exact), we can make use of cloud flexibility to scale up or scale down, and we can integrate with other features GCP offers.

My Moodle Tips #3

Andre Kho — Wed, 12 May 2021 05:37:36 +0000

Do full cleanup for every completed courses by your users (regularly or on demand). They usually don't aware of your limited resources.

From my perspective, you may need to:

Go to a Course settings > Reset course > Check all course module categories and options that considerably consume too many resources > Reset course
Unenroll inactive users. If you have external database enrolments and/or authenticated users, you may want to delete all associated user's data that not currently active from those tables. Make sure the related Moodle CRON jobs are active for this case.

Be aware of these course resources! (your case may be different)

Assignments
Files/Folders
Quizzes

My Moodle Tips #2

Andre Kho — Thu, 25 Feb 2021 14:08:31 +0000

If your Moodle starts consuming too much inodes available on a server, be sure to purge caches periodically.

Go to Site administration > Development > Purge caches, then click on Purge all caches.

It will relieve some spaces for you, especially if your resources are limited.

My Moodle Tips #1

Andre Kho — Fri, 15 Jan 2021 05:09:40 +0000

If your website going down and showing HTTP error code 500 and/or 503 or sometimes your website's template getting error in some places, you might want to check your opcache PHP extension.

Opcache has to be enabled. It will lower your resource usage significantly because of its opcode caching capability.

However, if it has always been enabled but causing errors like I mentioned above, you can disable opcache extension temporarily (like 3-5 seconds) then enable it again. Your PHP engine should be refreshed and your Moodle site will be up again (if your server doesn't refresh, restart it manually).

After 8 months 4 weeks and 1 day

Andre Kho — Fri, 18 Sep 2020 03:25:40 +0000

It has been a while I didn't post something for 8 months 4 weeks and 1 day 😂. What a shame.

Anyway, I just want to share my current state for now. There are a lot of things happened before and after the pandemic. My primary laptop broken (data were safe, hopefully), my only phone was also broken. However, I was able to work for developments and thesis since I still have to get to the campus and able to borrowed my sister's netbook (Thank God).

One of the great things was that me with my team can deliver new app using Flutter framework and deployed it successfully without too much drama (perhaps). The app was made for a campus which I work for and thankfully, it is the first project to be adequately documented. That is one achievement for our team by the way (a lot of do-not-tell-others stories here). Here is the app.

We still have to work and support on other things like e-learning system that our campus using. In other side, we still have to improve the way we work, managing projects and smaller tasks, and so on.

That's it for the post. Not a insightful post 😅. But anyway, sharing stories is good, no?

Introduction

Andre Kho — Fri, 20 Dec 2019 03:15:02 +0000

Helow, this is my first post and I'm gonna share my life story here from now on perhaps.

I'm Andre, from Indonesia, currently aged 22, Bachelor graduate in Informatics, Master student in Information System, working as IT support in an 'IT College'. Sounds great, never enjoy it 😄.

Why do I put that line? Stay tuned, because it's gonna be a long thread I will ever write in social platform like DEV. Life is tough, but I'm grateful with that 😤.

Good day! 😁

DEV Community: Andre Kho

Enable OAuth2 (XOAUTH2) for Sending Emails Using Gmail Account in OJS 3

Prerequisites

Step 1: Go to Google Cloud Console

Step 2: Create OAuth client ID

Step 3: Modify the get_oauth_token.php script

Step 4: Replace vendor packages manually

Step 5: Update Mail.inc.php dependency

Step 6: Retrieve a Refresh Token

Step 7: Add XOAUTH2 configuration to config.inc.php

Step 8: Test the mailing feature

Conclusion

Easy Uptime Monitoring Tool with Uptime Kuma in Google Cloud Platform

Create a Compute Engine instance with Container Deployment

Option #1, using TmpFS volume type

Option #2, using Disk as volume type

Configure Firewall rule

Deploy Laravel 8 to Compute Engine instance (LEMP stack)

Step 1: Create a Compute Engine instance

Step 2: Install and configure server environments

Step 3: Pull your app from a remote repository

Step 4: Install Laravel dependencies and configure project environment variables

Conclusion

Guide to Setup a Moodle website in Google Cloud Platform (GCP) with HTTPS

Post Updates

Assumptions 🤔

Overview 📑

Step 1: Provision a Compute Engine instance

Step 2: Install server environments

Set server time zone

Environment preparations

Check listening ports

Step 3: Make our subdomain/domain secure

Step 4: Get Moodle through Git

Step 5: Setup Moodle specific configurations

Copy installation to web root

Change web root entrypoint

Create new database and user

Update opcache and some PHP configuration

Continue installation from web interface

Change system path

Setup cron job for Moodle

Don't forget to revert Moodle directory permissions

Monitor the usage

Known Issues 🤯

Next steps ⏩

Opinion 💭

Conclusion 🏁

Further reading 📚

My Moodle Tips #3

My Moodle Tips #2

My Moodle Tips #1

After 8 months 4 weeks and 1 day

Introduction

Step 3: Modify the `get_oauth_token.php` script

Step 4: Replace `vendor` packages manually

Step 5: Update `Mail.inc.php` dependency

Step 7: Add XOAUTH2 configuration to `config.inc.php`