DEV Community: Claudio Fior

Javascript mixin

Claudio Fior — Thu, 11 Jul 2024 07:38:11 +0000

I'm an old PHP developer and I like traits.

I have to interact with a bank, the class in Bank that does login and add the traits to get and commit requests.

trait GetRequest {
   public function getRequest() {
      ...
   }
}
trait CommitRequest {
   public function commitRequest($data) {
      ...
   }
}
class Bank {
   use GetRequest;
   use CommitRequest;
   public function login() {
       ...
   } 
}

So I can split class traits in different parts.

I had to work on a Javascript project.
Ahhhh!!! The traits does not exists, what can I do?

Use the mixin


class Bank {
   login() {
       ...
   } 
}
let GetRequest = {
   getRequest() {
      ...
   }
}
let CommitRequest {
   commitRequest($data) {
      ...
   }
}
Object.assign(Bank.prototype, GetRequest);
Object.assign(Bank.prototype, CommitRequest);

Accessing Azure storage with a shared key in php

Claudio Fior — Fri, 28 Jun 2024 13:24:59 +0000

The new CTO like azure, so I had to access to azure storage blob and table from a php project.

The microsoft offical libraries are deprecated, so I made a the project caiofior/azure_storage using guzzle to access at this resources with a shared key.

$azureTable = new \Ftc\driver\AzureStorageTable(
    AZURE_STORAGE_ACCOUNT_TABLE_NAME,
    AZURE_STORAGE_ACCOUNT_TABLE_BASEURL,
    AZURE_STORAGE_ACCOUNT_TABLE_TOKENSAS
);

try {
    $tokenNata = $azureTable->get('file','%24filter='.urlencode('PartitionKey eq \''.$token.'\''));
} catch (\Exception $e) {
    die("Token errato !");
}

Example of sorage table access

    $azureBlob = new \Ftc\driver\AzureStorageBlob(
        AZURE_STORAGE_ACCOUNT_FILE_NAME,
        AZURE_STORAGE_ACCOUNT_FILE_BASEURL,
        AZURE_STORAGE_ACCOUNT_FILE_TOKENSAS
    );

   $azureBlob->put(
        'file',
        $_FILES['uploadFile']['name'],
        $_FILES['uploadFile']['type'],
        $_FILES['uploadFile']['size'],
        fopen($_FILES['uploadFile']['tmp_name'],'r')
   );

Example of file uploading

How to generate a shared key

Deploy changes of a php legacy application

Claudio Fior — Tue, 21 May 2024 15:21:15 +0000

The fuss

I am the mantainer of an 11 years old php application. It's a monolithic application, it has only 5 classes but it works. It was built using mysql 5 and php 5.3, now it keeps working with mariadb 10 and php 7.4.

Static code analisys

The most used tools to maintain and add new functionalities are:

git to trace changes in the code;
robot framework, to create test cases of the functionalities of the web application;
postman to test web servicies and api;
phpstorm and phpstan that provides a lot of suggestions to avoid deprecated functionalities and increase code readability;
sonarqube to trace the changes on code quality over time and find ugly pieces of code and forgotten parts.

Deploy the changes

At the moment the code deploy was made manually, comparing the code base with Beyond compare. Today I have tried a new way using git.

I have copied the .git folder in stage and production environment and keep only the stage and main branches in each of the two environments.

Every change is a new branch, when I have to publish the change i merge the branch with stage or main branch.

In the server I run

git fetch # Get the changes
git merge # Merge the changes

If there are some discrepancies in some script the last command shows a list of files not aligned.

I can restore the files with the command

git restore <<file_name>>

It works, but I have fear to create a complete mess.
Any suggestion of an alternative strategy?

Data in fixed fields text files

Claudio Fior — Fri, 19 Nov 2021 15:35:48 +0000

I have to read and write data from a fixed fields text file.
What is a it? It'a mess like this.

The data organized based on the character position:
1 - 11 Packet ID
12 - 11 Anagraph ID
23 - 10 Submission date
and so on.

How can I read the data in PHP?

First, ugly solution: use substr to read

 $identificativo_pacchetto = substr($line,0,11);
 $identificativo_mandante = substr($line,11,11);
 $data_affido = substr($line,22,10);
...

and str_pad to write

 $line = str_pad($identificativo_pacchetto,11);
 $line .= str_pad($identificativo_mandante,11);
 $line .= str_pad($data_affido,10);
...

It'a nightmare!!!

Lets'try with unpack and pack command.

One line of code to read.

$data =unpack('A11identificativo_pacchetto/A11identificativo_mandante/A10data_affido',$line));

and data is an array like this

{
   "identificativo_pacchetto":"1",
   "identificativo_mandante":"52301",
   "data_affido":"19-11-2021"
}

One line of code to write

$data = ['1','52301','19-11-2021'];
$line =pack('A11A11A10',...$data));

The is no need to pad the values.
Note that the are no filed names (eg . identificativo_pacchetto) and no separators (/).

I put all together defining the structure in an array.

$def = [
'A11identificativo_pacchetto',
'A11identificativo_mandante',
'A10data_affido
];
$data = ['1','52301','19-11-2021'];
$line = pack(implode('',preg_replace('/^(A\d+).*/','$1',$def)),...$data);
$data = unpack(implode('/',$def),$line);

More info:
PHP manual
PHP Cookbook
Stackoverflow

Aggregate data imported form an Excel file.

Claudio Fior — Mon, 11 Oct 2021 09:58:59 +0000

The users of an application make requests, identified by an id, relative one or more subjects.

The application produce a spreadsheet: the list of subjects and the id of the request.

The operators populate the spreadsheet with additional data for every subject and the import the file in the application.

The software has to aggregate the subject data in the requests identified by the request id and send the data to the users.

What is the best strategy to aggregate the spreadsheet data in requests?

This is at the moment my best solution.

Suggestions are welcome!

Decode data encrypted in Java in shell

Claudio Fior — Tue, 27 Apr 2021 10:32:03 +0000

I have to provide data available in an object storage encrypted by a Java application in a php site.

First of all I get the encrypted data using rclone.

It's an easy tick with the sell command

rclone cat "xda:pre-xda-data/001136/00004578..zip"

when rclone is properly configured.

Now the second step: decrypt the data. I was unable to use openssl: the password is encrypted in a completely different way.

At this point I wrote a small java program that received encrypted data from stdin and writes decrypted data to stdout.

import java.io.InputStream;
import java.io.IOException;
import javax.crypto.Cipher;
import javax.crypto.CipherInputStream;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.Charset;
import java.lang.IllegalArgumentException;
import java.security.NoSuchAlgorithmException;
import javax.crypto.NoSuchPaddingException;
import java.security.InvalidKeyException;

class Decode {
    public static void main(String[] args) throws NoSuchAlgorithmException, NoSuchPaddingException,InvalidKeyException,IOException {
        if (args.length != 1) {
            throw new IllegalArgumentException("Missing password");
        }
        String storageAesKey = args[0];
        byte[] raw = storageAesKey.getBytes(Charset.forName("UTF-8"));
        if (raw.length != 16) {
            throw new IllegalArgumentException("Invalid key size.");
        }
        SecretKeySpec key = new SecretKeySpec(raw, "AES");
        Cipher cipher = Cipher.getInstance("AES");
        cipher.init(Cipher.DECRYPT_MODE, key);
        InputStream input = new CipherInputStream(System.in,cipher);
        input.transferTo(System.out);
        input.close();
    }
}

At this point I can concatenate this two commands

rclone cat "xda:pre-xda-data/001136/00004578..zip" | java -cp /opt/javadecript Decode "password"

and execute the php passthru command

passthru ('rclone cat "xda:pre-xda-data/001136/00004578..zip" | java -cp /opt/javadecript Decode "password"');

Someone has a better idea?

Tools for creating shell command in PHP

Claudio Fior — Thu, 26 Nov 2020 16:54:22 +0000

I had to create a cli tool that reads excels file and push each row data to a web service.

It has to be called once a day using linux crontab.

Usually I create such software (middleware ?) from the scratch in PHP that is already present in the server.

I got many of these custom client applications and I would like to create a more structured application using a framework.

I tried Symfony console and it works, but I'm not happy.

I have 12 Mb of php scripts in vendor folder and 50 Kb of my code: an elephant that produce a mouse.

Moreover the server has php 7.0 so I have to use an old version on the framework (3.4).

When I will upgrade the server (next spring I suppose) I have fear to upgrade the framework to a more recent version (5.1) with a completely different logic.

What shall I do?

Any suggestion for a lightweight framework for a php cli command?

Thank you for your patience.

Extract data from a PDF

Claudio Fior — Tue, 14 Apr 2020 07:25:16 +0000

One of our providers give us some data ad PDF and I have to produce a JSON object for further elaborations.

For the textual information non problem: I used pdftotext to extract the text.

$content = shell_exec('pdftotext -enc UTF-8 -layout input.pdf -');

Then I used regular expressions to extract the data

 $anagrafica=array();
 if(preg_match('/^Denominazione\W*(.*)/m', $content, $aDenominazione)) {
     $anagrafica['denominazione']=$aDenominazione[1];
 }

How to extract the data of the semaphores that are images without labels?

I used the linux command pdftohtml

$rawImages = shell_exec('pdftohtml -enc UTF-8 -noframes -stdout -xml "'.$this->filePath.'" - | grep image');
$tok = strtok($rawImages,"\r\n");
while ($tok !== false) {
    $oImage = simplexml_load_string($tok);
    $images[]=$oImage;
    $tok = strtok("\r\n");
}

The output of pdftohtml in a xml document for each text box or image.

$rawImages is an array of the xml elements of the images ans I put them as SimpleXmlObjects in $images array.

Than I searched trough the array the images with 77 pixel of width and sort the by the vertical position.

The images are saved in the current directory of the script.

I queried the color of a pixel in a specific position of the image with convert command of ImageMagick library and saved the data in the JSON object.

$color = shell_exec('convert "'.$imagePath.'" -format \'%[pixel:p{100,50}]\' info:- ');
switch ($color) {
    case 'srgb(253,78,83)':
        $anagrafica[$this::chekcs[$pos]]='red';
    break;
    case 'srgb(123,196,78)':
        $anagrafica[$this::chekcs[$pos]]='green';
    break;
    case 'srgb(254,211,80)':
        $anagrafica[$this::chekcs[$pos]]='yellow';
    break;
};

At this point: is there an easy way to do the trick?

A suitable solution for php microservicies

Claudio Fior — Thu, 30 May 2019 13:35:21 +0000

I'm the maintainer of a old php web application birth in 2009 from the scratch (there is no framework, sic.) hosted in a Debian server.

It has been upgraded during the time, extended with new functionalities and now is a huge pile of code. But it works.

There is a plan of restyling the user interface, at least client side. This could be the chance to modularize the application that cover to many areas (relation with clients, contacts with providers, financial account, customer relationship management).

This could be the chance to convert an humongous application in a modular one.

One idea is: use microservicies!

The question is: what is the suitable solution in php?

At the moment I found these solutions:

1) ReactPHP, it is easy to install with Composer but requires a lot of dependencies (e.g. Symfony) to work;

2) Swoole PHP, it's a php module that must be installed with PECL. It's fast but not so easy to install in a Debian production server;

3) Amphp, it is easy to install with composer but has and does not require a lot of dependencies to work;

4) Web server that handles the requests on a different port and also catch PUT,DELETE requests.

Any suggestion that could be applied easily in a Debian machine?

The beautiful marriage of Python and MongoDB

Claudio Fior — Tue, 10 Apr 2018 09:05:42 +0000

I'm a PHP developer and managing big data stored with this language is not so easy.

Add data to the db is not a problem, but retrieving a summary and save it as csv is not to easy.

So this time I tried to use Python and it marvelous because:

In a moment you can create the object containing the aggregate conditions;
Saving the data as csv is easy.

#!/usr/bin/python
import pymongo
import csv
import collections
from pymongo import MongoClient
db = MongoClient().report_impresa
pipeline = [
    {
        "$group": {
            "_id": "$CompanySummary.CreditRating.CreditLimit",
            "count": {"$sum": 1},
            "min_rating": {"$min": "$CompanySummary.CreditRating.CommonValue"},
            "max_rating": {"$max": "$CompanySummary.CreditRating.CommonValue"},
            }
    },
    {"$sort": {"_id": 1}}
]
descr = db.ggs.aggregate(pipeline)
values = {}
for att in descr:
    chiave = att["_id"]
    if chiave is None:
        chiave = ""
    if chiave == "":
        values[""] = att
    else:
        values[int(chiave)] = att

with open('fido.csv', 'w') as csvfile:
    writer = csv.DictWriter(
                            csvfile,
                            fieldnames=[
                                "_id",
                                "count",
                                "min_rating",
                                "max_rating"
                                ]
                            )
    writer.writeheader()
    for k in sorted(values):
        writer.writerow(values[k])

Create ebook on the fly with php and pandoc

Claudio Fior — Fri, 23 Mar 2018 15:55:43 +0000

I'm digitalizing the Italian flora in the web site http://www.florae.it, but Internet is not everywhere so I produce and epub and I read it with Kobo ereader.

The trick is produce a text file formatted as Markdown

#### Sp. Phalaris canariensis {#t2108}

[Gen. Phalaris](#t1624)

[Scheda aggiornata - Segnala osservazione](http://www.florae.it/?id=2108)

Pianta glabra (!). Culmi ascendenti, glabri, avvolti fino in alto dalle guaine (le sup. sono decisam. rigonfie). Fg. molli, con lamina larga fino a 9 mm e ligula tronca, breve, di 2(3-5) mm; guaine scabre se strisciate dal basso verso l'alto. Pannocchia spiciforme ovale, densa 1-2 X 2-4 cm; sp.tte inf. spesso sterili; glume 6-7(9) mm, biancastre con linee longi-tud. verdi; lemma fertile 4-6 mm, pubescente; lemmi sterili di 3 mm circa.<br /><br />Macarones.<br />In tutto il terr.: C, però nell'It. Sett. per lo più avvent. incostante.<br /><br /><strong>Usi</strong> - Origin. delle Canarie e Medit. Occid. viene comunem. coltivata come alimento per i canarini; con la pulizia delle gabbie i semi finiscono facilmente tra le immondizie e possono quivi germinare dando luogo a brevi avventiziati. Viene pure usata come pianta ornamentale.

![Diffusione](/home/caiofior/Scaricati/florae.it//images/merged_map/0/c/0c7433833c7e9841626ea999f5f152db.png)

**Autore**
:   Linneo

**Nome Italiano**
:   Scagliola comune

**Ciclo riproduttivo**
:   ☉ Annuale

**Altezza**
:   3-5(7) dm

![](/home/caiofior/Scaricati/florae.it//images/merged_altitude/2/7/277819b553a78e6b2c28d9c898b60ff8.png)
**Fioritura**
:   da Aprile a Giugno

![](/home/caiofior/Scaricati/florae.it//images/merged_flower/b/4/b427ebc8ffe2221831c49fa57369653e.png)

and than call (pandoc)[https://pandoc.org/] form php. Something like this:

                    $cmd = 'pandoc --toc-depth=5 -t '.$format.' -f markdown+definition_lists+line_blocks -s -o '.$tempFileName.'.'.$format.' '.$tempFileName.' 2>&1';

                    $error = shell_exec($cmd);
                    unlink($tempFileName);
                    if (!is_file($tempFileName.'.'.$format)) {
                        throw new \Exception('Error on generating pandoc output with command '.$cmd.' '.$error,1509221630);
                    }
                    $fileName = $tempFileName.'.'.$format;

Where $tempFileName is the name of temporary Markdown file, $format is epub or other pandoc formats (doc, pdf, html) and $fileName the output file name.

At the moment I'm able to produce and epub of 150 Mb.

Data exchange between Excel and a LAMP server

Claudio Fior — Mon, 19 Mar 2018 10:01:22 +0000

Premise

Since 2011 the data collected in my company is stored and managed in an Excel spreadsheet. The company has grown up and now the data in exchanged in three offices in different parts of the Nation.

Since 2013 was adopted a solution based on a VPN.

Since 2017 the Excel macros have been transformed in a VB NET application.

Not they asked to realize a web application to manage the data.

How to feed the beast (the web application)

Optimal solution

The final solution will be a web service that exchanges the data from the web application that collects the client's data and the new application that store and manage the data.

Call to a web service for every insert/update in the Excel file

What to do in the meantime while the integration is not ready?

The VB NET application calls a web service to insert and update the data.

It's an easy trick creating a REST web service.

        Dim md5 As System.Security.Cryptography.MD5 = System.Security.Cryptography.MD5.Create()
        Dim request As System.Net.HttpWebRequest = System.Net.WebRequest.Create(Me.gatUrl & "?token=" & Me.GetMd5Hash(md5, token))
        request.AutomaticDecompression = System.Net.DecompressionMethods.Deflate Or Net.DecompressionMethods.GZip
        request.Method = "POST"
        request.ContentType = "application/x-www-form-urlencoded"
        request.Timeout = 5000


        Dim dataStream As IO.Stream = request.GetRequestStream()
        Dim postData As String = Newtonsoft.Json.JsonConvert.SerializeObject(rigaRichiesta)
        Dim enc As System.Text.UTF8Encoding = New System.Text.UTF8Encoding()
        Dim byteArray As Byte() = enc.GetBytes(postData)
        Using stream = request.GetRequestStream()
            stream.Write(byteArray, 0, byteArray.Length)
        End Using
        Dim responseString As String
        Try
            Dim result As System.Net.HttpWebResponse = request.GetResponse()
            Dim reader As New System.IO.StreamReader(result.GetResponseStream())
            responseString = reader.ReadToEnd()
            result.Close()
        Catch ex As Exception
            Me.Log.Text = "Errore nella chimata a " + Me.gatUrl + " " + ex.Message
            Me.Log.BackColor = Drawing.Color.Red

            Exit Sub
        End Try

The solution is slow: 10 minutes for 4k rows.

Call to a web service at Excel close passing all the data

Another solution is:

Save the Excel data as CSV;
Pass the CSV file as content of a REST call.

        Dim currentWorkbook As String = Me.workbook.FullName
        Dim currentFormat As Long = Me.workbook.FileFormat

        Dim nomeFile = currentWorkbook + ".csv"
        Me.foglioRichieste.SaveAs(nomeFile, Microsoft.Office.Interop.Excel.XlFileFormat.xlCSV)
        Me.foglioRichieste.SaveAs(currentWorkbook, currentFormat)

        Dim contenuto As String = System.IO.File.ReadAllText(nomeFile)

        Dim dt As DateTime = DateTime.Now
        Dim token As String = Me.token & dt.Year & dt.Month.ToString.PadLeft(2, "0") & dt.Day.ToString.PadLeft(2, "0")

        Dim md5 As System.Security.Cryptography.MD5 = System.Security.Cryptography.MD5.Create()
        Dim request As System.Net.HttpWebRequest = System.Net.WebRequest.Create(Me.gatUrl & "?azione=file&token=" & Me.GetMd5Hash(md5, token))
        request.AutomaticDecompression = System.Net.DecompressionMethods.Deflate Or Net.DecompressionMethods.GZip
        request.Method = "POST"
        request.ContentType = "application/x-www-form-urlencoded"
        request.Timeout = 50000

        Dim dataStream As IO.Stream = request.GetRequestStream()

        Dim enc As System.Text.UTF8Encoding = New System.Text.UTF8Encoding()
        Dim byteArray As Byte() = enc.GetBytes(contenuto)
        Using stream = request.GetRequestStream()
            stream.Write(byteArray, 0, byteArray.Length)
        End Using
        Dim responseString As String
        Try
            Dim result As System.Net.HttpWebResponse = request.GetResponse()
            Dim reader As New System.IO.StreamReader(result.GetResponseStream())
            responseString = reader.ReadToEnd()
            result.Close()
        Catch ex As Exception
            Me.Log.Text = "Errore nella chimata a " + Me.gatUrl + " " + ex.Message
            Me.Log.BackColor = Drawing.Color.Red

            Exit Sub
End Try

In 2 seconds the data has been updated in the server.

Bonus trick: saving the Excel file as CSV using Visual Basic produce a file of good quality, UTF8 encoded and that use coma as separator.