DEV Community: Yassine Elo

A small user messenger system using PHP & MySQL

Yassine Elo — Mon, 20 Mar 2023 20:00:14 +0000

Let's say you have authenticated users on your custom website, blog, community whatever, how cool would it be to let the user's chat with each other?
That's what I created this script for, a small and functional user messenger system in PHP & MySQL. Let me walk you through the script:

MySQL tables and how to make them work

These MySQL tables cover most of the program logic, as it should be, let the database do the work! After all everything we do is creating layers.

We create two tables for our chat system, one that saves all the active chats and the the other one to save the messages with a foreign key pointing to the userchat record ID.

A chat record is created before saving the first sent message. In our program logic we will save EVERY chat record and EVERY message record twice, that allows users to delete chats, without affecting the other end.

The tables:

-- Example table for our user accounts:

CREATE TABLE IF NOT EXISTS useraccount (
    id INT UNSIGNED NOT NULL AUTO_INCREMENT,
    user_name VARCHAR(64),
    PRIMARY KEY (id),
    UNIQUE (user_name) USING HASH
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;


-- User chat records:

CREATE TABLE IF NOT EXISTS userchat
(
    id INT UNSIGNED NOT NULL AUTO_INCREMENT,
    chat_owner INT UNSIGNED NOT NULL,
    chat_partner INT UNSIGNED NOT NULL,
    last_action INT UNSIGNED NOT NULL,
    created_at INT UNSIGNED NOT NULL,
    PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;


-- User chat message records:

CREATE TABLE IF NOT EXISTS userchat_msg
(
    id INT UNSIGNED NOT NULL AUTO_INCREMENT,
    chat_id INT UNSIGNED NOT NULL,
    msg_owner INT UNSIGNED NOT NULL,
    sender INT UNSIGNED NOT NULL,
    recipient INT UNSIGNED NOT NULL,
    msg_date INT UNSIGNED NOT NULL,
    msg_status TINYINT UNSIGNED NOT NULL,
    msg_text TEXT NOT NULL,
    PRIMARY KEY (id),
    FOREIGN KEY (chat_id) REFERENCES userchat (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

Example:

User A and user B have never texted before, or both deleted every chat from the past. For us that means there is no userchat record with user A as chat_owner and user B as chat_partner and vice versa.

Now User A (ID: 111) sends a "hello there" to user B (ID: 222), we need to create two records in the userchat table, one for each user as the chat owner, and two records in the userchat_msg table, both containing the original message text.

In PHP we need to run each query separately, and after each query get the userchat record ID, because we need to save it into the message records:

Add new userchat records like this:

-- Chat of user A with user B
$mysqli->query("INSERT INTO userchat (chat_owner, chat_partner, last_action, created_at) VALUES
(111, 222, UNIX_TIMESTAMP(), UNIX_TIMESTAMP());");

$myChatID = intval($mysqli->insert_id);

-- Chat of user B with user A
$mysqli->query("INSERT INTO userchat (chat_owner, chat_partner, last_action, created_at) VALUES
(222, 111, UNIX_TIMESTAMP(), UNIX_TIMESTAMP());");

$partnerChatID = intval($mysqli->insert_id);

And now that we initiated a new user chat for both users, we save the message "Hello there" that User A (ID: 111) sent to User B (ID: 222). Add new userchat_msg records like this:

$mysqli->query("INSERT INTO userchat_msg (chat_id, msg_owner, sender, recipient, msg_date, msg_status, msg_text) VALUES
($myChatID, 111, 111, 222, UNIX_TIMESTAMP(), 1, 'Hello there'),
($partnerChatID, 222, 111, 222, UNIX_TIMESTAMP(), 0, 'Hello there');");

The values for sender, recipient, msg_date and msg_text are of course the same. We saved the message message as 2 copies.
The differences are chat_id, msg_owner and msg_status. Last one because the copy of user A's sent message is set to 1 (opened) by default, and the the copy for user B is set to 0 (not opened), as long as user B hasn't opened the message, yet.

Each user will see only their own copy for all chats and messages, in their mailbox.

Now let's say after that user A sent another message "How are you?", add the new message like this:

$mysqli->query("INSERT INTO userchat_msg (chat_id, msg_owner, sender, recipient, msg_date, msg_status, msg_text) VALUES
($myChatID, 111, 111, 222, UNIX_TIMESTAMP(), 1, 'How are you?'),
($partnerChatID, 222, 111, 222, UNIX_TIMESTAMP(), 0, 'How are you?');");

Since there is already an active chat for and between user A and user B, we used the same chat IDs from before. Now we have to make sure, to update the column userchat.last_action

$mysqli->query("UPDATE userchat SET last_action = UNIX_TIMESTAMP() WHERE id = $myChatID OR id = $partnerChatID LIMIT 2;");

The column last_action holds the timestamp of the last sent message in that chat, thats why we update them for both chats. The user's mailbox will be ordered by last_action, so the newest chat is always on top.

Now we can create chats and send messages into that chat.

How to get the chat ID

Now as soon as users open their mailbox, they will get a list of all their active chats. That happens when our script messenger.php is requested.
If the operating user wants to open a chat or send a message to another user, in both ways the traget user name must be requested as a URL parameter partnerName.

The URL:

URL: myboard.com/messenger.php requests my mailbox
URL: myboard.com/messenger.php?partnerName=Franky requests the chat page with Franky

The chat page will show the message form and if a chat between me and the user I requested already exists, get the chat ID by user name and print the messages below.

Or if you want to rewrite the URL using HTACCESS:
URL: myboard.com/chat requests my mailbox
URL: myboard.com/chat/Franky requests the chat page with Franky

To achieve this you can use this .htaccess code:

RewriteEngine On
RewriteBase /server/httpdocs/myboardcom/

RewriteRule ^chat/?$ messenger.php [QSA]
RewriteRule ^chat/([A-Za-z\d]{3,64})/?$ messenger.php?partnerName=$1 [L]

This requires the user name to be unique and well formed, in our example a valid user name contains letters and numbers, case-insensitive and no spaces.
Minimum length of 3 is arbitrary and max. length should match the user name column in your useraccount table.

Advice: Even if you use htaccess and the regex pattern, you still have to validate the user name! Because the regex pattern in htaccess will be useless, if we request the page by messenger.php?partnerName=Franky.

How to get the chat ID now for real

We validate the requested user name and save it to $partnerName, we will use it to get the chat partner's user ID and user name (original case).
Because I can request chat/fRaNkY and Franky will be found, but I dont need that string to be displayed on the page.

As said before, this whole thing requires some user authentication system to work. We assume now the operating user's user ID is provided in $myUserID.

To get the chat partner by name and get the chat ID (if existant) use this QUERY:

SELECT usr.id, usr.user_name,
(SELECT cht.id FROM userchat cht WHERE cht.chat_owner = $myUserID AND cht.chat_partner = usr.id) AS chat_id
FROM useraccount usr WHERE usr.user_name = '$partnerName' AND usr.id != $myUserID LIMIT 1;

We retrieve the user ID and user name from the chat partner, and use a sub query to get an existant chat record. If I don't have a chat with the requested partner yet,
chat_id from the result above will be NULL.

The whole process:

// Let's just say the operating user is:

$myUserID = 111;

// Indicates if the requested user exists, if not show the mailbox page.
$openChat = FALSE;

if(isset($_GET["partnerName"])) {

    // Validate user name
    $partnerName = trim($_GET["partnerName"]);

    if(preg_match('/^[A-Za-z\d]{3,64}$/' , $partnerName))
    {
        // User name valid, get info from database:
        // Check if username really exists and get the user ID and the name from database
        // Also check if a chat between the operating and the requested user exists, if yes get the chat ID
        // RESULT: id, user_name, chat_id (will be NULL if no chat exists)

        $res = $mysqli->query("SELECT usr.id, usr.user_name,
        (SELECT cht.id FROM userchat cht WHERE cht.chat_owner = $myUserID AND cht.chat_partner = usr.id) AS chat_id
        FROM useraccount usr WHERE usr.user_name = '$partnerName' AND usr.id != $myUserID LIMIT 1;");

        if($res->num_rows === 1) {

            $row = $res->fetch_assoc();

            // User name of chat partner will be visible in the chat, don't use the $_GET value
            $partnerName = $row["user_name"];
            $partnerUserID = intval($row["id"]);

            // Will be int (0) if no chat exists, yet
            $myChatID = intval($row["chat_id"]);
            $openChat = TRUE;

            // If $openChat is TRUE, you can use these 3 variables: $partnerName, $partnerUserID, $myChatID (can be 0)

        }
        else {

            // Requested user name is valid but not existant!
            $errorFeedback = phpFeedback("error", "User name not found!");

        }

    }
    else {

        // else: invalid user name requested, you can redirect to 404 page here for example.
        $errorFeedback = phpFeedback("error", "User name does not exist!");

    }

}

Takeaway:
If $openChat is FALSE, the mailbox will be printed, with all user names as hyperlinks to open the chats.
If $openChat is TRUE, a chat page will be opened, if the requested user name exists.
Even if there is no chat yet, the HTML form for sending new messages will be printed.

Which means on every page request, we get the user name and ID of our chat partner, and my chat ID if I already have a chat with the partner.

Of course the chat partner has his own copy of the chat and his own chat ID, we need to get that ID upon sending a new message.

Now here is the fairly self explanatory script that lets the operating user send a message to the currently requested user.

if($openChat === TRUE AND isset($_POST["newMsgSubmit"]))
{

    // Validate and sanitize the text message
    // No empty input and no unnecessary empty lines, text messages like that are bad for the layout and user experience.

    if(!empty($_POST["newMsgText"]))
    {

        if(($msgText = trim($_POST["newMsgText"])) != "" )
        {

            // Sanitize line breaks, replace 3 or more line breaks in a row with 2 line breaks,
            // which is one free line in between the paragraphs.

            if($msgText = preg_replace('/(\n|\r|\r\n){3,}/', "\r\n\r\n", $msgText))
            {
                // Message is ready to be saved now, whe have our own chat ID in $myChatID, we need to get the $partnerChatID

                // $myChatID can be int(0), meaning the operating user has no open chat with the partner, but he could have deleted it and the the partner still has
                // his old chatID with all the message history
                // Remember: The whole chat history is saved two times, one copy for each user

                $partnerChatID = 0;

                $res = $mysqli->query("SELECT id FROM userchat WHERE chat_owner = $partnerUserID AND chat_partner = $myUserID LIMIT 1;");
                if($res->num_rows === 1) {
                    $row = $res->fetch_row();
                    $partnerChatID = intval($row[0]);
                }

                // Now create a new chat, if it doesn't exist, for myself and my partner
                $updateLastAction = FALSE;

                if($myChatID == 0 OR $partnerChatID == 0) $updateLastAction = TRUE;

                if($myChatID == 0) {
                    $mysqli->query("INSERT INTO userchat (chat_owner, chat_partner, last_action, created_at)
                    VALUES ($myUserID, $partnerUserID, UNIX_TIMESTAMP(), UNIX_TIMESTAMP());");

                    $myChatID = intval($mysqli->insert_id);
                }

                if($partnerChatID == 0) {
                    $mysqli->query("INSERT INTO userchat (chat_owner, chat_partner, last_action, created_at)
                    VALUES ($partnerUserID, $myUserID, UNIX_TIMESTAMP(), UNIX_TIMESTAMP());");

                    $partnerChatID = intval($mysqli->insert_id);
                }

                // Save messages now: One copy for each user
                $msgText = $mysqli->real_escape_string($msgText);
                $mysqli->query("INSERT INTO userchat_msg (chat_id, msg_owner, sender, recipient, msg_date, msg_status, msg_text)
                VALUES ($myChatID, $myUserID, $myUserID, $partnerUserID, UNIX_TIMESTAMP(), 1, '$msgText'),
                ($partnerChatID, $partnerUserID, $myUserID, $partnerUserID, UNIX_TIMESTAMP(), 0, '$msgText');");

                if($mysqli->affected_rows === 2) {
                    // Both messages saved successfully, if the userchat record was not created on point, you need to update the `last_action` column for the chats
                    if($updateLastAction) $mysqli->query("UPDATE userchat SET last_action = UNIX_TIMESTAMP() WHERE id = $myChatID OR id = $partnerChatID LIMIT 2;");
                }
                else {
                    // Saving error occured
                }
            }

        }

    }

}

Other queries:

That was basically it, opening chats and sending messages. Now this table structure we use allows us to do a bit more:

Count all my unread messages:

$query = "SELECT COUNT(id) FROM userchat_msg WHERE msg_owner = $myUserID AND recipient = $myUserID AND msg_status = 0;";

Deleting a chat:

Delete the chat messaages first, because of the foreign key constraint:

$mysqli->query("DELETE FROM userchat_msg WHERE chat_id = $myChatID AND msg_owner = $myUserID;");

$mysqli->query("DELETE FROM userchat WHERE id = $myChatID AND chat_owner = $myUserID LIMIT 1;");

Get my mailbox

We query all of our chats, with the name of the chat partners, the last action time stamp, and count unread messages in that chat with a sub query:

$res = $mysqli->query("SELECT cht.id, cht.last_action, usr.user_name,

    (SELECT COUNT(msg.id) FROM userchat_msg msg WHERE msg.chat_id = cht.id
    AND msg.recipient = $myUserID AND msg.msg_status = 0) AS numNewMsg

FROM userchat cht LEFT JOIN useraccount usr ON cht.chat_partner = usr.id

WHERE cht.chat_owner = $myUserID ORDER BY cht.last_action DESC LIMIT 100;");

Open a chat

Requesting a chat is simple:

$res = $mysqli->query("SELECT sender, msg_date, msg_text FROM userchat_msg
WHERE chat_id = $myChatID AND msg_owner = $myUserID ORDER BY msg_date DESC LIMIT 100;");

But after requesting it, make sure to mark all messages as seen after printing them to screen.

$mysqli->query("UPDATE userchat_msg SET msg_status = 1 WHERE chat_id = $myChatID AND msg_owner = $myUserID AND msg_status = 0 LIMIT 100;");

And that's it, let me know what you think.

PHP: Multiple image file upload with dynamic file storage

Yassine Elo — Thu, 16 Mar 2023 02:45:57 +0000

I wrote a simple but wholesome file upload script in PHP, for beginners.
You can upload multiple image files at once, they will be saved dynamically inside automatically created subfolders based on the date.
The file sources will be logged in a mysql table and for every saved image, also a thumbnail sized version will be created using PHP's extension IMAGICK.

Let's break it down, we will use these two files:

config.php - MySQL connection, file storage, file limitations and validation function
index.php - File upload form, file saving process and gallery of uploaded images

Remember: Obviously this is not for public usage, you need some kind of user authentication to run this script. For beginners: If you are not sure how to code a user authentication in PHP,
there are lots of tutorials and I can recommend this one: https://alexwebdevelop.com/user-authentication/

`config.php` - MySQL connection, file storage, file limitations and validation function

MySQL file log table

For every uploaded and successfully saved image file, insert a new record, that makes it easier to manage the files later.

The file log table:

CREATE TABLE IF NOT EXISTS myfiles (
    id INT UNSIGNED NOT NULL AUTO_INCREMENT,
    src_original VARCHAR(100) NOT NULL,
    src_thumb VARCHAR(100) DEFAULT NULL,
    uploaded_at INT UNSIGNED DEFAULT NULL,
    file_ext VARCHAR(5),
    media_type VARCHAR(50),
    PRIMARY KEY (id)
);
-- myfiles (id, src_original, src_thumb, uploaded_at, file_ext, media_type)

The file source is stored as a relative path, for example: "uploads/2023/02/08/image.jpg"

File storage

Relative file paths, define file base

All files sources will be stored inside our "uploads" directory. And all file paths will begin with "uploads":

The example from before: "uploads/2023/02/08/image.jpg" this will be stored in mysql as the file source.
That can result in an URL like: mywebsite.com/images/uploads/2023/03/image.jpg

The directory "image" inside the web root folder is therefore our file base. This file base must be defined, at the beginning of our config.php file.

$fileBase = "images";

Dynamic file storage

Now we don't want every uploaded file to be stored in the same folder, thats why we need a function that checks on demand, if today's subfolder exists and if not, tries to create it.

We just use PHP's date() function to create subfolder names.

// Dynamic uploads storage:
// Based on year, month and day
// e.g. "uploads/2023/04/14"

$fileBase = "images";

// Create/get current file storage path
// Returns string on success, or FALSE if directory doesnt exist and cant be created

function getCurrentFileStorage():string|false {
    if(!is_dir($GLOBALS["fileBase"])) return FALSE;

    // Our globally defined file base
    $base = $GLOBALS["fileBase"] .'/';
    // Our uploades folder
    $fs = 'uploads';

    // We need to return the relative file path, keep it separated
    if(!is_dir($base . $fs)) {
        if(!mkdir($base . $fs)) return FALSE;
    }

    // Year based file storage
    $fs .= date("/Y");
    if(!is_dir($base . $fs)) {
        if(!mkdir($base . $fs)) return FALSE;
    }

    // Month based file storage
    $fs .= date("/m");
    if(!is_dir($base . $fs)) {
        if(!mkdir($base . $fs)) return FALSE;
    }

    // Day based file storage
    $fs .= date("/d");
    if(!is_dir($base . $fs)) {
        if(!mkdir($base . $fs)) return FALSE;
    }

    return $fs; // return relative file path
}


// Call the function on upload submit,
// so that subfolders are created only when they are needed

if(isset($_FILES["myFile"]["name"]))
{
    if(!$fileStorage = getCurrentFileStorage()) {
        die("File storage not available");
    }

    // Upload process ...
}

Of course this function can be changed to save files based on year only, or on weeks etc.
The possibilities are bound by PHP's date() function parameters.

File limitations

Now we want to provide security and protect ourselves from potential malicious files, for example if we let users upload image files on our website, too.
What we do is not 100% secure for the only reason that there is no such thing as 100% security. The data of an image file can be manipulated to bypass certain validation steps.
For example to execute harmful code disguised as a jpg file.

A simple way to raise security is using restrictions, that means before saving an uploaded file, we will check each file if it passes the limitations and will not allow anything else.

The file limitations are specified in our config file as such:

$fileMaxSize = 1024 * 1024 * 10; // 10 MB

// How many uploaded files will be saved at once?
//If the user uploads 17 files, only the first 15 will be saved.
$fileMaxUploads = 15;

// Size in pixel of the thumbnail image that will be created using IMAGICK
$thumbnailSize = 150;

Now we define a whitelist array, with all supported file extensions as array keys pointing to their respective mime/media type.

// whitelist = array ("extension" => "media type")
$mediaWhiteList = array("jpg" => "image/jpeg",
"jpeg" => "image/jpeg",
"gif" => "image/gif",
"bmp" => "image/bmp",
"png" => "image/png",
"webp" => "image/webp");

If the uploaded file does not match the criteria above, it will not be saved. The validation happens next.

`index.php` - File upload form, file saving process and gallery of uploaded images

At the start we include our config file and define a little function to collect error messages.
Since you're uploading multiple files, you can get different responses, for clean code we collect all error messages in one variable and display them later.
If one uploaded file doesn't pass validation, add one error message and continue with the next file.

require_once "config.php";

// UPLOAD PROCESS RESPONSE
// Feedback messages appended in this string, using the function below
$uploadResponse = "";

function addUploadResponse($class, $text):void {
    $GLOBALS["uploadResponse"] .= "<p class=\"$class\">$text</p>\r\n";
    return;
}

// Example, attach file name to error message:
addUploadResponse('error', $_FILES["fileUpload"]["name"] . ' file type not supported');

And then, if an submit action happened, we start our file saving process.
Let's break it down step by step:

if(isset($_FILES["fileUpload"]["name"]))
{
    // On upload submit, check if file storage is available

    if($fileStorage = getCurrentFileStorage())
    {

Count uploaded files

Don't save all submitted files! Control the amount with $fileMaxUploads from config.php
Any files after max. value will be omitted!

$numFiles = count($_FILES["fileUpload"]["tmp_name"]);

if($numFiles > $fileMaxUploads) {
    addUploadResponse("info", "You can only upload $fileMaxUploads files at once");
    $numFiles = $fileMaxUploads;
}

Start the loop

Validate each file of $_FILES["fileUpload"]. If a validation step fails, add an error message and continue with the next file.
Our files in this array can be iterated by the 3rd array index.
More info: https://www.php.net/manual/en/features.file-upload.multiple.php

// $_FILES["fileUpload"]["tmp_name"][0] => First uploaded file
// $_FILES["fileUpload"]["name"][0]     => Original name of first uploaded file
// $_FILES["fileUpload"]["size"][0]     => Size of first uploaded file
// etc.

Inside the loop:

Check for PHP upload errors

more info: https://www.php.net/manual/en/features.file-upload.errors.php

if($_FILES["fileUpload"]["error"][$i] != 0) {
    addUploadResponse("error", $_FILES["fileUpload"]["name"][$i] . " Error: File can not be saved");
    continue;
}

Validate file extension

Get the file extension from the original file name with our function getFileExtension() from config.php.
It extracts the file extension from the original file name and returns it as a lowercase string, unless the extension is not whitelisted, in which case it returns FALSE.

function getFileExtension($name):string|false
{
    $arr = explode('.', strval($name)); // split file name by dots
    $ext = array_pop($arr); // last array element has to be the file extension
    $ext = mb_strtolower(strval($ext));
    // Return file extension string if whitelisted
    if(array_key_exists($ext, $GLOBALS["mediaWhiteList"])) return $ext;
    return FALSE;
}

And the validation step inside the loop:

if(!$ext = getFileExtension($_FILES["fileUpload"]["name"][$i])) {
    addUploadResponse("error", $_FILES["fileUpload"]["name"][$i] . " - file type not supported");
    continue;
}

Validate file size

if($_FILES["fileUpload"]["size"][$i] > $fileMaxSize) {
    addUploadResponse("error", $_FILES["fileUpload"]["name"][$i] . " - file too big");
    continue;
}

Validate content type

Check if the uploaded file has the content type it should have according to our whitelist.

if($mediaWhiteList[$ext] != mime_content_type($_FILES["fileUpload"]["tmp_name"][$i])) {
    addUploadResponse("error", $_FILES["fileUpload"]["name"][$i] . " - invalid media type");
    continue;
}

Do a Byte Signature Check according to file type

Use our checkMagicBytes() callback function from config.php.
Read more in this post

if(!checkMagicBytes($ext, $_FILES["fileUpload"]["tmp_name"][$i])) {
    addUploadResponse("error", $_FILES["fileUpload"]["name"][$i] . " - invalid file type");
    continue;
}

File is valid now

Create the source path to be stored and a random filename, then move the uploaded file and prepare the info to be saved later in mysql.

// Random file name:
$filename = bin2hex(random_bytes(4)) . '.' . $ext;

// Source path:
$srcOriginal = $fileStorage .'/'. $filename;

if(!move_uploaded_file($_FILES["fileUpload"]["tmp_name"][$i], $fileBase .'/'. $srcOriginal)) {
    addUploadResponse("error", $_FILES["fileUpload"]["name"][$i] . " - failed to save file");
    continue;
}

// Info to be saved in mysql later
$logImages[$i] = array("srcOriginal" => $srcOriginal,
"srcThumb" => NULL,
"ext" => $ext,
"type" => $mediaWhiteList[$ext]);

Create a thumbnail sized version with IMAGICK

The PHP extension "Imagick" is available on most hosting providers and servers these days and can be installed if not.
The Imagick class provides super easy functions to crop images into a square by using
Imagick::cropThumbnailImage(150, 150) to create a thumbnail with 150px. However in the comments below the PHP manual,
you can find information on how to crop GIF images. The contributed comments in the manual are GOAT sometimes.

try {
    $objIMG = new Imagick(realpath($fileBase .'/'. $srcOriginal)); // New Imagick object made of uploaded file
    $objIMG->setImageFormat($ext);

    // Special case for cropping gifs - https://www.php.net/manual/en/imagick.coalesceimages.php

    if($ext == "gif") {
        $objIMG = $objIMG->coalesceImages();
        foreach ($objIMG as $frame) {
            $frame->cropThumbnailImage($thumbnailSize, $thumbnailSize);
            $frame->setImagePage($thumbnailSize, $thumbnailSize, 0, 0);
        }
        $objIMG = $objIMG->deconstructImages();
    }
    else {
        $objIMG->cropThumbnailImage($thumbnailSize, $thumbnailSize);
    }
}
catch (ImagickException $e) {
    addUploadResponse("info", $_FILES["fileUpload"]["name"][$i] . " - failed to create thumbnail image<br>" . $e->getMessage());
    continue;
}


// SAVE NEW THUMBNAIL IMAGE NOW:
$srcThumb = $fileStorage . "/th" . $filename;

if(file_put_contents($fileBase .'/'. $srcThumb, $objIMG)) {
    // Update thumbnail source in image log:
    $logImages[$i]["srcThumb"] = $srcThumb;
}
else {
    addUploadResponse("error", $_FILES["fileUpload"]["name"][$i] . " - failed to save thumbnail image");
}

$objIMG->clear(); // free memory usage

And that's our file loop.

Save files to MySQL

At the end of our loop, if images have been saved successfully the $logImages array is set and contains the file info to be stored.

if(isset($logImages)) {
    $savedFiles = 0;
    $time = time();
    $stmt = $mysqli->prepare("INSERT INTO myFiles (src_original, src_thumb, uploaded_at, file_ext, media_type) VALUES (?,?,?,?,?);");

    foreach($logImages as $log) {
        $stmt->bind_param("ssiss", $log["srcOriginal"], $log["srcThumb"], $time, $log["ext"], $log["type"]);
        $stmt->execute();
        if($mysqli->affected_rows === 1) {
            $savedFiles++; // Count successfully saved files
        }
    }
    $stmt->close();

    if($savedFiles > 0) {
        addUploadResponse("success", $savedFiles . " Files uploaded!");
    }
    else {
        addUploadResponse("error", "No files saved");
    }
}

And finally our HTML upload form

Remember to use brackets [] in the name attribute of the input file element,
this way PHP will provide the multiple files as an array inside $_FILES
Also the keyword multiple

<form class="gridForm" action="index.php" method="post" enctype="multipart/form-data">
    <h4>upload files from your device</h4>
    <?php echo $uploadResponse; ?>
    <input type="file" id="inputFile" name="fileUpload[]" multiple>
    <input type="submit" id="inputSubmit" name="uploadSubmit" value="upload now">
    <p><a href="index.php">refresh</a></p>
</form>

And that is it. You can get the whole source code on my personal repository.

PHP: Magic Bytes check for image files

Yassine Elo — Sun, 05 Mar 2023 20:04:08 +0000

This is based on my previous post about writing boolean functions that do a byte signature check depending on the file extension.

The Byte Signature of a file can be manipulated too and also it may be redundant to use it for validation, because the URL to a corrupt image file on your website will simply not be displayed by the browser. (And in your upload script you should whitelist file extensions anyway!)

But for the sake of completion I want to share all the 5 functions I wrote, to validate jpg, png, gif, bmp and webp image files.

<?php
// function from my previous post
$ext = getFileExtension($_FILES["myFile"]["name"]);


// Magic Bytes validation callback functions:
$imgCallBackFuncs = array("jpg" => "magicBytesJPG",
"jpeg" => "magicBytesJPG",
"gif" => "magicBytesGIF",
"bmp" => "magicBytesBMP",
"png" => "magicBytesPNG",
"webp" => "magicBytesWEBP");


function checkMagicBytes($ext, $file):bool
{
    if(!isset($GLOBALS["imgCallBackFuncs"][$ext])
    OR !function_exists($GLOBALS["imgCallBackFuncs"][$ext])) {
        return FALSE;
    }
    return call_user_func($GLOBALS["imgCallBackFuncs"][$ext], $file);
}

// The 5 different image types:

function magicBytesBMP($file):bool
{
    if(!$handle = fopen($file, 'r')) return FALSE;
    if(!$readBytes = fread($handle, 2)) return FALSE;

    // Byte Signature for BMP:
    // "42 4D" (2 Bytes)
    if(mb_strtoupper(bin2hex($readBytes)) == "424D") {
        return TRUE;
    }
    return FALSE;
}

function magicBytesPNG($file):bool
{
    if(!$handle = fopen($file, 'r')) return FALSE;
    if(!$readBytes = fread($handle, 8)) return FALSE;

    // Byte Signature for PNG:
    // "89 50 4E 47 0D 0A 1A 0A" (8 bytes)
    if(mb_strtoupper(bin2hex($readBytes)) == "89504E470D0A1A0A") {
        return TRUE;
    }
    return FALSE;
}

function magicBytesGIF($file):bool
{
    if(!$handle = fopen($file, 'r')) return FALSE;
    if(!$readBytes = fread($handle, 6)) return FALSE;

    // Byte Signature for GIF:
    // GIF87a "47 49 46 38 37 61"
    // GIF89a "47 49 46 38 39 61"
    $readBytes = mb_strtoupper(bin2hex($readBytes));
    if($readBytes === "474946383761"
    OR $readBytes === "474946383961") {
        return TRUE;
    }
    return FALSE;
}

function magicBytesWEBP($file):bool
{
    if(!$handle = fopen($file, 'r')) return FALSE;
    if(!$readBytes = fread($handle, 12)) return FALSE;

    // Byte Signature for WEBP:
    // "52 49 46 46 ?? ?? ?? ?? 57 45 42 50" (12 Bytes)
    $readBytes = mb_strtoupper(bin2hex($readBytes));
    if(preg_match("/52494646[A-F0-9]{8}57454250/", $readBytes)) {
        return TRUE;
    }
    return FALSE;
}

function magicBytesJPG($file):bool
{
    if(!$handle = fopen($file, 'r')) return FALSE;
    if(!$readBytes12 = fread($handle, 12)
    OR !$readBytes4 = fread($handle, 4)) return FALSE;
    fclose($handle);

    /*
    // Byte Signature for JPG:
    1 - FF D8 FF DB (4 bytes)
    2 - FF D8 FF E0 00 10 4A 46 49 46 00 01 (12 bytes)
    3 - FF D8 FF EE (4 bytes)
    4 - FF D8 FF E1 ?? ?? 45 78 69 66 00 00 (12 bytes)
    5 - FF D8 FF E0 (4 bytes)
    */
    $readBytes12 = mb_strtoupper(bin2hex($readBytes12));
    $readBytes4 = mb_strtoupper(bin2hex($readBytes4));

    if($readBytes4 == "FFD8FFDB" OR $readBytes4 == "FFD8FFEE"
    OR $readBytes4 == "FFD8FFE0" OR $readBytes12 == "FFD8FFE000104A4649460001"
    OR preg_match("/FFD8FFE1[A-F0-9]{4}457869660000/", $readBytes12)) {
        return TRUE;
    }
    return FALSE;
}

PHP File Upload: Check uploaded files with magic bytes

Yassine Elo — Sun, 05 Mar 2023 04:09:57 +0000

In this post I want to describe my thought process from when I wrote a PHP script to upload image files to my website.

It is super easy to save an uploaded file with PHP, all the information about the new files is provided inside the superglobal array $_FILES

Now, I want to ensure my script is safe to use, so that for example another authenticated user besides myself can upload files to the website, without having to fear a malicious file is placed in the server by a malicious user to accomplish whatever malicious goal.

I started to get scared tbh, because I was shocked about how little control you have as a web developer, when a user sends a file to the web server. In fact, I learned that you can not even prevent a user from sending a file to the webserver. So it comes down to the question: How do I validate the uploaded file?

I gave it a lot of thought and I came to the conclusion that you can always provide security for your website, by using restrictions. The more I restrict the properties of the uploaded file, the more security I get, or so I suppose.

Let's do that with image files.

So in my PHP script I grab all the information about the image file just uploaded and check them against my set limitations.
The goal is to make sure the image file is what it claims to be.

1. File Size

$maxFileSize = 1024 * 1024 * 10; // Max. 10 MB
$maxImgSize = 4000; // Max. 4000px (for width and height)

I'll admit, I would rather not restrict those too much, I believe every smartphone today shoots images with at least 2000px and 3-4 MBs and that may be played down.

2. File Extension And Mime/Media Type

I want to allow only jpg, png, gif, bmp and webp images to be uploaded, I think that covers enough ground.

In this array I set the file extensions together with their respective mime/media types.

$imgWhiteList = array("jpg" => "image/jpeg",
"jpeg" => "image/jpeg",
"gif" => "image/gif",
"bmp" => "image/bmp",
"png" => "image/png",
"webp" => "image/webp");

Now I use the following function to get the file extension from the uploaded file's name. If the given file extension is not whitelisted, this function returns FALSE. So calling this function is also a validation step. The code is self explanatory:

function getFileExtension($name):string|false
{
    // split file name by dots
    $arr = explode('.', strval($name));
    // last array element has to be the file extension
    $ext = array_pop($arr); 
    $ext = mb_strtolower(strval($ext));
    // Return file extension string if whitelisted
    if(array_key_exists($ext, $GLOBALS["imgWhiteList"])) {
        return $ext;
    }
    return FALSE;
}

if(!$ext = getFileExtension($_FILES["file"]["name"])) {
    die("Invalid file type");
}

// $ext is now your file extension
// Check the mime type like this:

if($imgWhiteList[$ext] != mime_content_type($_FILES["file"]["tmp_name"]))
{
    die("Invalid media type");
}

But do I really need to check the mime media type?

At this point my researching started to intensify and I came to understand that you can detect extension and media type of a file,
but it is still data that can be manipulated. And additionally the PHP function mime_content_type() uses the "magic.mime" file in your PHP installation to determine the file.

I didn't know what that is.

The official PHP documentation and the top comments below enlightened me a bit:
https://www.php.net/manual/en/function.mime-content-type

But I still am not sure how reliable this technique really is.

But I'm not giving up, I want to have a safe and reliable upload script, even when I have to use magic!

3. The Magic Bytes

So while I was tirelessly googling things like "help how to protect against virus file php upload" I eventually came across something often referred to as Magic Numbers or Magic Bytes. As it seems binary files all contain a kind of signature. The "real" type of a file is readable inside its first few bytes. Now the length of this signature varys between different file types. But luckily we live in the age of knowledge and wikipedia has this to offer:

https://en.wikipedia.org/wiki/List_of_file_signatures

Now to be quick, I made a plan:

Take the signature info of the desired file type from wikipedia
Read the first few bytes of the uploaed file and check it against the signature from wikipedia
Write boolean functions that do this and use them finally to check if the file is what it claims to be.

This is an example for gif images. According to wikipedia a gif image file has to start with a byte signature of 6 bytes:

Either "47 49 46 38 37 61" (GIF87a)
or "47 49 46 38 39 61" (GIF89a)

So the trick is simple, for GIF files read the first six byte and check if they are either one of the values above.

function magicBytesGIF($file):bool
{
    if(!$handle = fopen($file, 'r')) return FALSE;
    if(!$readBytes = fread($handle, 6)) return FALSE;

    $readBytes = mb_strtoupper(bin2hex($readBytes));

    if($readBytes === "474946383761"
    OR $readBytes === "474946383961") {
        return TRUE;
    }
    return FALSE;
}

BOOM. A solid boolean function to check if the uploaded image.gif is REALLY a gif and not a -
I don't even know what I am protecting myself against, I just dont want my website to be hacked.
Oh my god reading up on Cyber Security can make a man paranoid!

Anyways.

Here is another example with JPG, now jpg files can have 5 different byte signatures, as following:

1. "FF D8 FF DB" (4 bytes)
2. "FF D8 FF E0 00 10 4A 46 49 46 00 01" (12 bytes)
3. "FF D8 FF EE" (4 bytes)
4. "FF D8 FF E1 ?? ?? 45 78 69 66 00 00" (12 bytes)
5. "FF D8 FF E0" (4 bytes)

So I need to read 4 bytes and 12 bytes, and then check if the read bytes match with any of the 5 above.

In variant 4 the question marks mean it can be any value in that position. Don't ask me why, I feel like deciphering ancient writings. But I shall not be scared, this is solved with simple regex.

function magicBytesJPG($file):bool
{
    if(!$handle = fopen($file, 'r')) return FALSE;
    if(!$readBytes12 = fread($handle, 12)
    OR !$readBytes4 = fread($handle, 4)) {
        return FALSE;
    }
    fclose($handle);

    $readBytes12 = mb_strtoupper(bin2hex($readBytes12));
    $readBytes4 = mb_strtoupper(bin2hex($readBytes4));

    // It must be one of these:
    if($readBytes4 == "FFD8FFDB" OR $readBytes4 == "FFD8FFEE"
    OR $readBytes4 == "FFD8FFE0"
    OR $readBytes12 == "FFD8FFE000104A4649460001"
    OR preg_match("/FFD8FFE1[A-F0-9]{4}457869660000/", $readBytes12)) {
        return TRUE;
    }
    return FALSE;
}

Now one might ask, why do I need to read 4 bytes and 12 bytes each once? At the beginning I just read the first 20 bytes or something of each file, regardless which type and tried to compare the values with the known signatures.

But I got results which were to me equally unexpected and confusing.

It turns out the byte values (or at least the hexa decimal translations) change depending on how much bytes you read.

I do not know why, and a Engineer may laugh at me now, but hey I am still learning!

One last thing before we finish: Bitmaps are fairly uncomplicated.

function magicBytesBMP($file):bool
{
    if(!$handle = fopen($file, 'r')) return FALSE;
    if(!$readBytes = fread($handle, 2)) return FALSE;

    // file signature bitmap "42 4D" (2 Bytes always)
    if(mb_strtoupper(bin2hex($readBytes)) == "424D") {
        return TRUE;
    }
    return FALSE;
}

Now I learned a new technique how to better control the files that are being uploaded to the server.

I'm curious what everyone else thinks, are the magic bytes useful?

You may now think, what the hell did I just read?! Was that a tutorial, a presentation or a elaborated question?

I'll tell you, it was my first blog post. Hope you enjoyed.