DEV Community: Christian

PHP 8 is amazing. I still prefer C#.

Christian — Fri, 08 Jan 2021 12:00:00 +0000

With the end of 2020 came the release of PHP 8 and also the release of .Net 5.0.0, the former also being discussed on HackerNews (and here, here or here). I write a lot of code in PHP, while frequently using C# with .Net Framework and recently more often .Net Core – maybe half/half. I admire PHP and I think PHP 8 proves that the folks designing and developing PHP care much about the language and the future of PHP in general. In this blog post, I would like to show why I think PHP matters and why I still prefer C#.

What’s new in PHP 8?

First, there’s named arguments, so you can much more easily use functions that have a lot of arguments or just to make arguments more clear. As the PHP 8 release announcement shows:

htmlspecialchars($string, double_encode: false);

PHP 8 also supports class/method attributes which may replace the often-used PHPDoc to configure behaviour for classes or methods. They look a bit silly with using #[] as syntax, but it looks like it’s for backwards compatibilty (as # introduces a one-line comment).

#[Route("/api/posts/{id}", methods: ["GET"])] public function foo()

Then there’s also constructor property promotion, which removes the need to set properties of classes which are arguments of the constructor manually in the constructor. Then there’s union types which allow to pass more than a single type to a strictly typed method. There are match expression, which remove the need to write a switch statement. And there’s a nullsafe operator, which allows to navigate through class methods which may be null:

$country = $session?->user?->getAddress()?->country;

There are also som other improvements under the hood like Just-in-time compilation, a lot of stuff around type system and the error handling and various syntax tweaks. Basically, PHP 8 is a major release in respect to the language design and less regarding the functionality (expect maybe for JIT-compilation). It fixes a lot of legacy that was around for so long and allows the language to grow – and even get better.

Is PHP really worth it?

Yes, absolutely. PHP is an amazing language by providing an easy entry, simple deployment, widely available good webhosters which support it. It is stable, reliable and usually just works. PHP is often laughed at for it’s language design. However, PHP has a vibrant community and one of the largest publicly available package repository. PHP is easy to install, easy to use and quite easy to deploy as it consists only of files that need to be copied to a webserver – there’s no platform-specific compilation.

The most often heard arguments against PHP are it’s sloppiness/laxiness around types (script types are not mandatory), it’s forgiveneess (as long as the syntax is ok, semantic is “adaptable”) and it’s bad language design (way to much legacy).

Then why prefer C#?

I’ll show you some arguments why I prefer the language design of C# over the one of PHP. For context: I mostly use both languages for web-based things. So it’s always usually JSON in and JSON out.

Good dependency management

While I think PHP does the job (it does it very well), I began to like C# (especially with .Net Core). I mostly admire the design of the C# language. Composer, PHP’s semi-official package manager, is one of the best solutions for dependency resolution. It is perfectly capable resolving transitive dependencies which make’s it very easy for package maintainers to define their requirements and makes it very easy for package users to require some package and validate whether it’s acutally compatible with the other dependencies. In .Net, assembly binding and transitive dependencies often caused trouble, issues and build problems. With .Net Core, this got a lot easier having a simpler dependency-definition format.

Strict typing and compile-time syntax validation

One of the quirks of being an interpreted language like PHP is that syntax validation is either done statically or when actually running the code. This can lead to issues that get uncovered quite late (at best when running automated test, at worst in production). With C# being a compiled language, at least the syntax is validated when compiling. This does not mean that your code magically works always, it just means that the most basic issues are catched early.

// This is valid syntax in PHP. Does it make sense? No. function foobar() : void {} $val = foobar();

Obviosuly, your preferred IDE may point out that this code does not make sense. But that’s not the point. In a large codebase it gets very hard to keep eyes on all such issues.

Streams

PHP does not have streams available. It does have resources which are some kind of special variable that hold a reference to an external resource. If you search on how to how to read an image (or any file) from a remote host and write it to a local file, you’re pointed to something like this:

$imageString = file_get_contents("http://example.com/image.jpg"); $save = file_put_contents('Image/saveto/image.jpg',$imageString);

C# uses some more code for that, but then: What type does $imageString actually have? It is not a resource, it is a string. And if you guess, you’ll want to load a 100 MB file from a remote server, how much memory do you need? At least 100 MB. Then I simply prefer using some derivate of the Stream class which allows me to read, write, seek and all that using a quite low memory footprint. Having worked on projects which do image processing on webservers with often quite large images (up to 50 or sometims 100 MB), Streams are a welcome gift.

Generics, Enums and all the missing parts of a nice language

PHP actually does have a proposal to implement enums at a future point. It also does have a four year old proposal for implementing generics. Do you need enums and generics to get the job done? Absolutley not. Does it make the job easier? Absolutely yes. Especially, with basically every database providing an enum datatype, it would be nice to have an enum in PHP and not use a string for that. There are great libraries for that, and they work quite well. And that brings me do my last argument:

Proper type deserialization

Searching the web for how to “deserialize in php to a custom class”, the first link that Stackoverflow that pops up, proposes something like that:

$data = json_decode($json, true); $class = new Foobar(); foreach ($data as $key => $value) $class->{$key} = $value;

Obviously, this does not work with nested classes, strict types (or somehow it does?, because of the lax typing in PHP). Other proposals loke much more fancier, or suggest using stdClass (the base class of every class) and the use PHPDoc type-hints to hint the type to the IDE. Then I prefer the C# way which is perfectly capable of deserializing nested objects into their proper type.

FooObj object = JsonConvert.DeserializeObject<FooObj>(jsonString);

In todays world, with distributed (or micro-) services, it is possible that some data is passed from one service to another. The receiving service gets some JSON string and deserializes that string into a shared class definition. With that, you get script types for free and modifying one side (modifying the contract) also let’s you track the change for the receiving service. Without proper deserializing into actual class types, you’re just guessing a lot.

Whould I still use PHP?

Yes, for sure. With Symfony, one of if not the most advanced PHP frameworks, it’s easy to write webservices. With Doctrine, an ORM, it’s quite easy to read/write to databases and map objects (However, I still prefer something like a passive record pattern). PHP has a large, active community and a lot of open source projects which help you in many cases. PHP is absolutely evoling and I think if it keeps going in the same direction as it did with PHP 8, it may have a position in the language landscape for a long time.

PHP has started of in a (web-) landscape that was so different from what we know today. PHP kept (and still keeps) a lot of legacy around, which can also be a good thing (others don’t, looking at .Net ecosystem and their many UI frameworks) but also can be so bad that, at some point, you will break half of the internet by making the language incompatible with WordPress.

Conclusion

With .Net Core 3 and, the latest release, .Net Core 5, Microsoft shows that it cares aboud it’s .Net plaform and it shows that it is perfectly capable to compete with well-known languages for web APIs like PHP, Python and Java. PHP 8 proved, that the team around PHP is willing to improve the language in significant ways and to cut old tails off. I think PHP 8 is amazing and it does it’s job perfectly. However, I think C# just provides the better language design for todays api-based world.

This blog post was originally posted on my companys blog under https://en.globalelements.ch/2021/01/08/php-8-is-amazing-i-still-prefer-c/.

Backing up MySQL in AWS using a read-replica

Christian — Sat, 11 Apr 2020 16:35:55 +0000

Background

We have several databases on AWS using RDS (Relational Database Service). We would like to backup those databases automatically. AWS provides „Snapshots“ for that, but they have several limits (like we can’t use them to restore a local database and it’s a bit harder to manage hundreds of them). Backing up a database is basically a simple task. However, we want to back-up the database with as least impact as possible to users (customers) of this database, so we can’t just connect to it and run a backup command.

TL;DR

We use the AWS CLI and IAM to create a read replica of an existing RDS DB instance (in this case MySQL). We use IAM to authenticate against this RDS instance and then backup databases using a pre-defined list. After we completed the backup, we remove the read replica. The backup is compressed as .tar.gz. The whole process is automated using bash.

Prerequisites

For this to success, you need available

a RDS instace with IAM authentication enabled
a bash terminal (we use a lot of Ubuntu/Linux these days)
the AWS CLI (v1 is enough)
jq for JSON parsing (see https://stedolan.github.io/jq/ )
the RDS instance has a user that can authenticate with the AWSAuthenticationPlugin (see https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.DBAccounts.html).
an IAM role that is able to create a read replica of the database you want to backup and that can connect to the read replica using IAM (see https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.IAMPolicy.html )

Step 0: Environment

I’ll use several environment variables through this tutorial. I’ll assume you have the AWS CLI configured and the IAM user has enough access rights to create a read replica and delete it again afterwards.

# The unique identifier of the RDS read replica
RDS_REPLICA_IDENTIFIER=
# The unique identifier of the RDS instance to backup
RDS_SOURCE_IDENTIFIER=
# The username which the IAM user is allowed to use, i'll use backup here
DB_USER=
# The endpoint will be the hostname of the read replica
ENDPOINT=

Step 1: Create a read replica

First, we want to create a read replica using the AWS CLI. We make the replica publicly available so that we can access it from our computer. Use $RDS_REPLICATE_IDENTIFIER for the unique name of the replica and $RDS_SOURCE_IDENTIFIER for the unique name of the instance you want to backup. We’ll add a tag here, but that’s not necessary. You might also want to consider to upsize/downsize your read replica depending on the size of your master instance (a powerful replica might cost you not that much but might improve backup speed).

aws rds create-db-instance-read-replica \
        --db-instance-identifier $RDS_REPLICA_IDENTIFIER \
        --enable-iam-database-authentication \
        --source-db-instance-identifier $RDS_SOURCE_IDENTIFIER \
        --publicly-accessible \
        --tags Key=purpose,Value=backup

Step 2: Wait until the replica is ready

It will take some time until the read replica is ready and can be accessed using the MySQL CLI. So let’s use some bash magic and wait until the read replica has the „available“ state. We use jq here to extract the status from the JSON response of the AWS CLI.

STATUS=creating
while [["$STATUS" != "available"]]
do
        sleep 30
        echo " Checking..."
        STATUS=$( aws rds describe-db-instances \
                --db-instance-identifier $RDS_REPLICA_IDENTIFIER | jq -r ".DBInstances[0].DBInstanceStatus" )
        echo " -> Status is ${STATUS}"
done

If that went through smooth (probably after 5 to 10min), let’s move forward and find the information needed to connect to our instance.

Step 3: Describe the endpoint

We will need at least a hostname, a username, a password and a port to connect to the MySQL read replicate. So let’s find those informations. I’ll use again jq to extract the information from the JSON response of the AWS CLI. The last line generates a temporary access token for our IAM user, so that we can backup the database. Be careful, the token expires after 900s (15min), so if your backups take longer you might want to make sure you can renew the token.

ENDPOINT=$( aws rds describe-db-instances \
         --db-instance-identifier $RDS_REPLICA_IDENTIFIER )
DB_HOST=$( echo $ENDPOINT | jq -r ".DBInstances[0].Endpoint.Address" )
DB_PORT=$( echo $ENDPOINT | jq -r ".DBInstances[0].Endpoint.Port" )
DB_PASS=$( aws rds generate-db-auth-token \
   --hostname ${DB_HOST} \
   --port ${DB_PORT} \
   --region eu-central-1 \
   --username ${DB_USER} )

Step 4: Doing the backup

Now we will be able to backup our databases. Let’s again use some bash magic to dump not only one but several databases using mysqldump.

DATABASES=(
    # add/remove database names as you wish
    "db1"
    "db2"
)
for DATABASE in "${DATABASES[@]}"
do
    echo "-- Exporting database <${DATABASE}>"
    mysqldump \
            --enable-cleartext-plugin
            -h${DB_HOST} \
            -u${DB_USER} \
            -p${DB_PASST} \
            --port=${DB_PORT} \
            --databases $DATABASE | \
            sed -e 's/DEFINER[]*=[]*[^*]*\*/\*/' > ${RDS_SOURCE_IDENTIFIER}_${DATABASE}.sql
    echo " -> Dump ${DATABASE} complete, wait a few seconds..."
    # Experience tells, let's wait a few secs or you'll get errors
    sleep 3
done

This script will dump the databases from our RDS instance one-by-one and store them in a SQL file in the current directory. Depending on your use-case you might want to configure mysqldump with one of its many options. There are quite some important options there to make your backups both fast, reliable and complete.

Also, don’t forget to compress the sql files and move them somewhere safe. We’ll usually upload them to S3/Glacier, a very cost-effective storage. As sql files are plain-text, they are very small when compressed and this will not cost very much.

tar -czvf "${RDS_SOURCE_IDENTIFIER}.tar.gz" ./*.sql

Step 5: Removing the read replica

Last but not least, let’s remove the read replica so it won’t incurr any more costs to our AWS account. We can remove the read replica with the same command as we would remove a regular RDS instance. So be careful here and make sure your IAM user does not have rights to remove the master but only the replica. We use the same logic to wait for the delete to be finished as above. Make sure to skip final snapshots, as they are not available to read replicase (but the API expects you to provide the option).

aws rds delete-db-instance \
        --db-instance-identifier $RDS_REPLICA_IDENTIFIER \
        --skip-final-snapshot

STATUS=deleting
while [ "$STATUS" == "deleting" ];
do
        sleep 30
        echo "   Checking..."
        STATUS=$( aws rds describe-db-instances \
                --db-instance-identifier $RDS_REPLICA_IDENTIFIER || \
                echo '{"DBInstances":[{"DBInstanceStatus":"gone"}]}' | \
                jq -r ".DBInstances[0].DBInstanceStatus" )
        echo "   -> Status is ${STATUS}"
done

The AWS CLi will fail as soon as the instance is gone, so we’ll trick the script by returning some dummy JSON to indicate that the instance is gone.

Conclusion

Backup is an important task and should be done properly. I demonstrated how to automate backing up an AWS RDS (MySQL) instance using the AWS CLI and some bash. I’ll used a read replica to keep the impact on the production system low as the backup happens from a „detached“ database. This is a very cost-effective and performant option.

Going further, I would recommend detaching the read replica (making it independent from the master) and to add some better error handling. I also did not demonstrate how to backup those files (we usually move them to S3/Glacier), as this is dependent on your needs, your storage options and also cost.

Why use Infrastructure as a Code

Christian — Mon, 18 Mar 2019 00:00:34 +0000

I believe that Infrastructure-as-a-Code (also known as IaaC) becomes a central aspect of future infrastructure management. In this article I will outline some advantages and disadvantages of IaaC.

Advantages of IaaC

First, I believe that documentation which is asynchronous to the task itself tends to get outdated and ignored. If your job requires you to document a change in a wiki page (or similar to that), you might easily forget to track that change. This results in differences between effective and assumed implementation, which can result in service failures, security or compliance issues.

Second, I believe in automatization. Automating regular tasks makes executing them easier, safer and more resilient. Auomatization requires a structured definition of the required output, and by using IaaC the output is defined in some form of code. Automatization makes it easier to compare a desired state with an effective state, and usually computers tend to be more error-prone than humand.

And third, I believe in disaster recovery. A backup is useless, if one never tried to restore it. The same applies to infrastructure. Knowing that a brand-new and up-to-date infrastructure is just a click away, one can have a disaster recovery in place, at no additional cost.

But there are also some counterpoints to IaaC.

Disadvantages of IaaC

The first point is quite obvious: One size does not fit all. Having a heterogenous IT landscape, there might not be a tool available fitting all purposes and hardware sets one might have. Most IaaC toolsets tend to support „The Cloud“ only, so not having your application in the cloud might break the idea of using IaaC for everything.

Second, merging existing processes is not easy. Assuming you have all your applications already in the cloud, extracting the information from your provider and generating the sources required for IaaC is something almost all toolsets can do for you. What none of the toolsets can do is fitting easily into established processes and procedures. They not only might require some adjustment, they require it for sure. And having to automate a complex process is not going to be easy.

Providers

There are several providers and toolsets for IaaC. We at Global Elements believe in the strenghts of Terraform, one of the earliest providers of IaaC. With Terraform, you write code in a domain-specific language that is very close to JSON.

Like virtually all IaaC providers, Terraform uses something called „state“. The state describes the last applied version of the infrastructure definition. It is used to compare the latest applied state, the desired state and the effective state. This allows to make sure that manual changes are detected (and then overriden) by applying the desired state to the infrastructure. Using this process, manual changes are automatically overriden. This requires an engineer to always write changes to the Terraform definition.

Terraform has support for almost any of the well-known Cloud providers, including Amazon, Microsoft, Google, Cloudflare, DigitalOcean and quite more.

Summary

By using Infrastructre-as-a-Code, you receive a structured and automatically documented definition of your current infrastructure. You can detect (and override) manual changes by applying a desired state to the current state. By this process you will make sure, that everyone is forced to use the defined IaaC toolset. IaaC is nothing new, but it is still not yet widely adapted. There are some disadvantages when having hybrid infrastructure, but IaaC toolsets provide still more advantages.

An Introduction to Graylog

Christian — Mon, 07 Jan 2019 20:50:08 +0000

Graylog, recently released in version 2.5, is an alternative to the well-known ELK stack (Elasticsearch, Logstash, Kibana). In comparison to the ELK-stack, Graylog uses MongoDB as a storage backend for settings and authentication, and leverages Elasticsearch as a document store.

This post is going to be a part of a series that will explore Graylog in detail. Stay tuned!

The sample dashboard as shown in the graylog documentation.

If you’re looking for an easy-to-go application, that is yet quite powerful and can be customized quite well – and on top of that, is Open Source – Graylog might be your solution. Additionaly compared to the „classic“ ELK stack, Graylog provides a fully-fledged authentication backend, and also allows to integrate with any LDAP directory (for example, ActiveDirectory).

The key concept of Graylog are inputs, which are nothing else than definitions of „how to receive messages“. It supports the well-known Syslog format and the GELF format, which is a JSON-definition maintained by Graylog itself. GELF is supported through UDP and TCP, which makes Graylog quite powerful – delivering log messages through the internet is not an issue at all, as the TCP connection does support TLS for encrypted transfer. Graylog can also easily be configured to act as a relay and forward any (or messages matching a pattern) to another instance.

Inputs are routed into streams, which represent a collection of messages. Streams can be configured to be filled by messages matching a pattern (e.g. a regular expression). If you ever require to extract information from a log message, extractors come to help. Extractors allow to, well, extract data from a messages by applying regular expressions, and the converting the data to various formats, like date or IP-adresses.

If that is not enough, Graylog provides a concept called pipelines. Pipelines basically allow you to „code“ a custom, complex process on how an incoming log message might be processed. This can include modifying and routing a message. A classic example would be that a message is routed into a stream based on an IP address, but the IP address must be removed from the message before it is stored (e.g. any GDPR compliance).

Having implemented Graylog in multiple projects, we would love to assist you on your next project requiring a scalable, centralized and powerful logging application.

Der Beitrag An Introduction to Graylog erschien zuerst auf Global Elements GmbH.