loading...
Cover image for Distributing Databases from The Main Server to The Gitlab Private Repository using Integromat.

Distributing Databases from The Main Server to The Gitlab Private Repository using Integromat.

darkterminal profile image Imam Ali Mustofa ・4 min read

Experiments

In the first phase of the experiment, I want to create a Rest Server using ReactPHP and make the Gitlab Private Repository as data storage. However, I cannot write it down and explain it, because it is not finished yet.

On this occasion I would like to share something that I think can be useful for other developers.

Required:

After everything is prepared, what you need to pay attention to is the endpoints that will interact with Integromat and distribute the database to the destination location.

In this case I am using Codeigniter Framework to create my website and create a Rest API endpoint to interact with Integromat, let's just say this is Webhook (hmm... looks stupid). I will provide a snippet from the Controller that will interact with the Integromat and Library FlySystem as data distributors for Gitlab.

Web App + Rest API

Database Operation endpoint as a data distribution for gitlab that handles Backup Database and uploads Zip Files to the Gitlab Private Repository.

Here's the snippet and I save it in the application/controllers/webmaster/Database_operation.php


<?php

/**
 * Database Opertaion
 */
class Database_operation extends Betta_Controller
{

    public function __construct()
    {
        parent::__construct();
    }

    public function backup()
    {
        $status = false;
        try {
            //load helpers
            $this->load->helper('file');
            $this->load->library('zip');
            $this->load->library('flysystem');

            //load database
            $this->load->dbutil();

            //create format
            $db_format = array('format'=>'zip','filename'=>'backup.sql');
            $backup = $this->dbutil->backup($db_format);

            // file name
            $dbname='backup-on-'.date('Y-m-d H:i').'.zip';
            $save = FCPATH . 'db/' .$dbname;

            // write file
            write_file($save,$backup);
            file_put_contents( FCPATH . 'db/backup.log', $dbname);

            // Upload & Commit to Gitlab
            $this->flysystem->write( $dbname, $backup );
            $status = true;
        } catch (Exception $e) {
            $status = false;
        }

        $this->response([
            'status' => $status,
            'type' => ( $status ) ? 'success' : 'error',
            'msg' => ( $status ) ? 'Database berhasil di backup dengan nama: ' . $dbname . ' Pada tanggal: ' . date( 'Y-m-d H:i:s' ) : 'Gagal Backup Database!',
            'waktu' => date( 'Y-m-d H:i:s' ),
            'file' => $dbname,
            'sumber' => base_url(), 
            'destinasi' => 'gitlab'
        ], Betta_Controller::HTTP_OK, 'json');
    }

    public function remove_backup()
    {
        $status = false;
        $last_log = file( FCPATH . 'db/backup.log' );
        $last = end($last_log);

        // Hapus file backup
        if ( unlink(FCPATH . 'db/' . $last) ) {
            $status = true;
        } else {
            $status = false;
        }

        $this->response([
            'status' => $status,
            'type' => ( $status ) ? 'success' : 'error',
            'msg' => ( $status ) ? 'File backup berhasil dihapus' : 'Gagal Hapus File Backup Database!'
        ], Betta_Controller::HTTP_OK, 'json');
    }
}


On the controller there is a FlySystem library that I integrate with the website application that I made. And following the library's appearance, I save the file in application/libraries/Flysystem.php

<?php

use League\Flysystem\Adapter\Local as LocalAdapter;
use League\Flysystem\Filesystem;
use RoyVoetman\FlysystemGitlab\Client;
use RoyVoetman\FlysystemGitlab\GitlabAdapter;
use TCB\Flysystem\Sync;

defined('BASEPATH') OR exit('No direct script access allowed');

/**
 * Flysystem
 */
class Flysystem {
    /**
     * @var mixed
     */
    private $filesystem;
    /**
     * @var mixed
     */
    private $ci;
    /**
     * @var mixed
     */
    private $client;
    /**
     * @var mixed
     */
    private $adapter;

    public function __construct() {
        $this->ci = &get_instance();
        $this->client     = new Client('GITLAB_PERSONAL_ACCESS_TOKEN', 'PROJECT_ID', 'PROJECT_BRANCH', 'GITLAB_URL');
        $this->adapter    = new GitlabAdapter($this->client);
        $this->filesystem = new Filesystem($this->adapter);
    }

    /**
     * Write File
     * @param  string $path     location of a file
     * @param  string $contents file contents
     * @return boolean true|false
     */
    public function write($path, $contents) {
        return $this->filesystem->write($path, $contents);
    }

    /**
     * Update content of file
     * @param  string $path     location of a file
     * @param  string $contents file contents
     * @return boolean           success boolean
     */
    public function update($path, $contents) {
        return $this->filesystem->update($path, $contents);
    }

    /**
     * Read file
     * @param  string $path location of file
     * @return string       file contents or false on failure
     */
    public function read($path) {
        $contents = $this->filesystem->read($path);

        return $contents;
    }

    /**
     * @param $path
     * @return mixed
     */
    public function has($path) {
        $contents = $this->filesystem->has($path);

        return $contents;
    }

    /**
     * @param $path
     * @return mixed
     */
    public function delete($path) {
        $contents = $this->filesystem->delete($path);

        return $contents;
    }

    /**
     * @param $from
     * @param $to
     * @return mixed
     */
    public function rename($from, $to) {
        $contents = $this->filesystem->rename($from, $to);

        return $contents;
    }

    /**
     * @param $from
     * @param $to
     * @return mixed
     */
    public function copy($from, $to) {
        $contents = $this->filesystem->copy($from, $to);

        return $contents;
    }

    /**
     * @param $path
     * @return mixed
     */
    public function deleteDir($path) {
        $contents = $this->filesystem->deleteDir($path);

        return $contents;
    }

    /**
     * @return mixed
     */
    public function action() {
        return $this->filesystem;
    }

    /**
     * Sync It helps you sync 2 directories at a time. There are two types.
     * @param  [type] $from [description]
     * @param  [type] $to   [description]
     * @return [type]       [description]
     */
    public function sync($from) {
        $remote = $this->filesystem;
        $local  = new Filesystem(new LocalAdapter($from));

        $sync = new Sync($local, $remote, '/');
        $sync->sync();

        return [
            'write_on_remote' => $sync->getUtil()->getWrites(),
            'delete_on_local' => $sync->getUtil()->getDeletes(),
            'update_on_both'  => $sync->getUtil()->getUpdates(),
        ];
    }
}

And now it's time to explain the flow of the scenario on the Integromat that I have made as a cover above.

This time I assume that you have made the above preparations.

Integromat Scenario

HTTP Module #1

This HTTP Module is responsible for making requests to the main server and backing up with the endpoints according to the controller above: https://your_website/webmaster/database-opeation/backup which is only done in a certain period of time according to our settings.

Integromat Router #2

Integromat router will distribute the response to several subsequent modules as recipients of messages or reports.

Telegram Bot #3

Telegram Bot will receive a message from the response received on the first request and then send a message to the Telegram Channel consisting of clients. So the client will get the status of every backup activity carried out by the server.

HTTP Module # 4

Then between the Telegram Bot module and the HTTP Module there is a condition filter, if the Backup is successful the HTTP Module will delete the backup file stored on the main server via the endpoint https://your_website/webmaster/database-operation/remove-backup, because the backup file is successfully distributed to the Gitlab Private Repository.

Google Sheet Module # 5

After everything is done successfully then Integromat will record a report of every backup activity to Google Spreasheet.

Done!

Are you interested in trying it out?

Thanks to:

Posted on by:

darkterminal profile

Imam Ali Mustofa

@darkterminal

Founder of Betta Dev Indonesia - Software & Application Developer "The best moment in my life is.. when I encountered an error and I can fix it!"

Discussion

markdown guide