DEV Community: hbgl

PHP now has support for undefined!?

hbgl — Sun, 04 Feb 2024 18:27:37 +0000

Before you dismiss this post as some cheap clickbait, please take a look at the following code and tell me if this is PHP:

<?php

function greeting(string $name, string|null|undefined $title = undefined) {
    $greeting = 'Hello';
    $greeting .= match ($title) {
        null => ' ',
        undefined => ' ??? ',
        default => " $title ",
    };
    $greeting .= $name;
    return $greeting;
}

echo greeting('John Doe');

Drumroll… it actually is perfectly valid PHP code.

I know what you are thinking: “This is not JavaScript. PHP does not have undefined.” And you would be correct. This is where I need to qualify my previous statement. The above code may be valid PHP but it does not execute successfully. Turns out, PHP really does not know about undefined.

Fatal error: Uncaught Error: Undefined constant “undefined”

— The PHP runtime

But no worries, we can fix this minor problem with only two lines of code.

<?php

enum undefined { case undefined; }
define('undefined', undefined::undefined);

How it works

To make the function greeting() work, we need two things:

a) First, we need to have a type called undefined for the type annotation of the function parameter. An enum will do. A single enum case is required, otherwise we would not be able to do anything with it. The name of the case is irrelevant.

b) Second, we need a constant called undefined for the default argument value. The syntax function ($x = undefined) may look unfamiliar, but all it says is that the default value for $x is equal to the constant named ‘undefined’. The built-in function define() lets us create a global constant with that name.

What’s missing?

We have no language support for undefined whatsoever. Meaning, neither the null-safe nor null-coalescing operators work like they do in JavaScript:

<?php

enum undefined {
    case undefined;
    public function foo() {
        return 'Foo';   
    }
}
define('undefined', undefined::undefined);

var_dump(undefined ?? 'Fallback'); // Prints enum(undefined::undefined)

$obj = undefined;
var_dump($obj?->foo()); // Prints string(3) "Foo"

Wrapping up

Should you introduce this pseudo language extension into your production code base? Definitely not. In the end it is just minor syntactic sugar for referencing an enum case.

While I do think that singleton symbol-like objects may sometimes come in handy (see example), it is better to be explicit about it.

The post PHP now has support for undefined!? appeared first on hbgl.

Constants are no longer constant in PHP

hbgl — Wed, 31 Jan 2024 20:14:28 +0000

I really hope that I am not opening a can of worms here because I think I have found something in PHP that I should not have. Something that should not exist: const mutable objects.

Yes, you read that correctly. Since PHP 8.1, any regular old object can be a constant, even mutable ones. Why is this a big deal? Well, because constants used to be constant and never change. This is no longer the case.

Here, have a quick taste:

<?php

class MyClass {
    public int $value = 1;
    public function set(int $value) { $this->value = $value; }
}

const MY_OBJECT = new MyClass();
// Equivalent to:
// define('MY_OBJECT', new MyClass());

echo MY_OBJECT->value . "\n";
MY_OBJECT->set(2);
echo MY_OBJECT->value . "\n";

// Prints:
// 1
// 2

Interesting.

A short history lesson
If you carefully read the documentation for the define() function in PHP, you will come across the following statement about constant values:

In PHP 5, value must be a scalar value (int, float, string, bool, or null). In PHP 7, array values are also accepted.

https://www.php.net/manual/en/function.define.php

Objects were clearly not intended to be constant values. While there was never an official reason given for this restriction, some argued that it was due to objects being inherently mutable.

With the addition of enums in PHP 8.1, it became possible to use enum cases as constant values. A very useful feature, indeed. Although the documentation for define() was never amended, the following code works as expected:

<?php

enum Suit {
    case Clubs;
    case Diamonds;
    case Hearts;
    case Spades;
}

const CLUBS1 = Suit::Clubs;
define('CLUBS2', Suit::Clubs);

This brings us back to the topic at hand: objects.

Enums are classes, and enum cases are objects

The PHP documentation tells us how enums ought to be classified:

The Enum itself is a class, and its possible cases are all single-instance objects of that class. That means Enum cases are valid objects and may be used anywhere an object may be used, including type checks.

https://www.php.net/manual/en/language.enumerations.overview.php

The second sentence is especially interesting. If an enum case can be used anywhere an object can be used, then is the inverse also true? Remember, enum cases can be used as constants. So what about objects?

<?php

class MyClass {
    public $foo = 123;
}

const MY_OBJECT = new MyClass();

In PHP 8.0 and earlier this code would terminate with an error, but starting with 8.1, it runs just fine.

Constant objects are a now thing (for better or worse).

Defining constant objects

We have already seen two ways to create constant objects: the declare() function and the const keyword at the top level. Both are functionally identical.

<?php

const MY_OBJECT1 = new MyClass();
declare('MY_OBJECT2', new MyClass());

Class constants, however, cannot be declared this way.

<?php

class Foo {
    // Fatal error: New expressions are not supported in this context
    public const MY_OBJECT = new MyClass();
}

Declaring a class constant requires a constant expression, which new MyClass() is not. But do you know what is a constant expression? Another constant! Meaning, we can declare object constants on the class level by reusing global constants.

<?php

const GLOBAL_MY_OBJECT = new MyClass();

class Foo {
    public const MY_OBJECT = GLOBAL_MY_OBJECT;
}

Interestingly, there seems to be a difference between const on the top level and const on the class level that is not mentioned in the documentation.

Using constant objects

When it comes to reading access and method calls, constant objects can be used like any regular object.

<?php

class MyClass {
    public $foo = 123;
    public function getFoo() { return $this->foo; }
    public function setFoo($value) { $this->foo = $value; }
}

const MY_OBJECT = new MyClass();

MY_OBJECT->foo;         // Read property.
MY_OBJECT->getFoo();    // Call function.
MY_OBJECT->setFoo(321); // Call mutating function.

Two things that are off limits, though, are assigning to a property (or offset) and creating a reference to a property.

<?php

// Fatal error: Cannot use temporary expression in write context
MY_OBJECT->foo = 456;
MY_OBJECT[0] = 789;
$x = &MY_OBJECT->foo;

This is not a limitation on the mutability of the object itself. As stated earlier, we can mutate the object through method calls. Rather, it is a limitation of the PHP compiler. From the compiler’s point of view, assigning to a property of a constant object is equivalent to assigning to a property of a temporary object.

<?php

// Fatal error: Cannot use temporary expression in write context
(new MyClass())->foo = 1;

To work around this restriction, we can assign the constant to a regular variable or return the constant from a function.

<?php

$myObject = MY_OBJECT;
$myObject->foo = 1;

function getMyObject() { return MY_OBJECT; }
getMyObject()->foo = 2;

The constant() function may also come in handy as it returns any constant by name, thereby circumventing the ‘temporary expression’ problem.

What do we take from it?

My suggestion is: do not use constant objects (enums aside).

They are counter-intuitive (due to their mutability) and also completely unnecessary. Plus, I am pretty sure that constant objects were never an intended feature in the first place. My theory is that they snuck into the language with the introduction of enums.

It would be for the best if constant objects got removed from the language.

The post Constants are no longer constant in PHP appeared first on hbgl.

Elegant immutable object pattern in PHP

hbgl — Sat, 27 Jan 2024 22:13:38 +0000

As many of you know, immutability is an extremely useful concept that makes code more predictable and generally easier to understand. Case in point, in PHP we have the mutable DateTime as well as DateTimeImmutable, the latter being recommended over the former.

And why is that?

Well, being an immutable object means that it can be passed around fearlessly without worrying about someone mutating it. No more guessing whether you should make a defensive copy or not. Plus, the API of DateTimeImmutable is not any worse than that of DateTime, so you get all of the benefits basically for free.

But it does not stop with DateTimeImmutable. We were always able to write our own immutable class types in PHP, and with the recently introduced readonly properties (PHP 8.1) and readonly classes (PHP 8.2) it is now easier than ever.

Let me show you a quick example of an immutable Address class:

<?php

readonly class Address {
    public function __construct(
        public string $name,
        public string $line1,
        public string $line2,
        public string $city,
        public string $state,
        public string $zipCode,
        public ?string $phone,
    ) {}
}

$myAddress = new Address(
    'John Doe',
    '4711 Main St',
    'Apt 4',
    'Anytown',
    'CA',
    '90210',
    null
);

How to update properties

Having immutable objects is nice and all but in the real world people do change their addresses. In the context of immutable objects, we “update” properties by creating new versions of existing objects. So instead of mutating the existing object, we create a new object that is a copy except for the properties we want to change.

Here is what it would look like if we wanted to add a phone number to the address:

<?php

$myAddress = new Address(
    'John Doe',
    '4711 Main St',
    'Apt 4',
    'Anytown',
    'CA',
    '90210',
    null
);

$myNewAddress = new Address(
    $myAddress->name,
    $myAddress->line1,
    $myAddress->line2,
    $myAddress->city,
    $myAddress->state,
    $myAddress->zipCode,
    '(555) 555-1234'
);

While the code is easy to understand, it is also quite verbose. It took 9 lines to update a single property since all properties had to be passed into the constructor. This problem gets even worse the more properties a class has. It is not hard to imagine that this would be a deal breaker for some, and a reason to fall back to mutable objects.

Fortunately, we can do a lot better. Let me show you an alternative method that updates a property in only a single line of code:

<?php

$myNewAddress = $myAddress->with(phone: '(555) 555-1234');

// We can also change two properties.
$myOtherAddress = $myAddress->with(
    city: 'Sometown',
    phone: '(555) 555-6789',
);

Pretty neat, right? To understand what is going on, let’s take a look at the implementation.

<?php

enum Unchanged {
    case Value;
}

readonly class Address {
    public function __construct(
        public string $name,
        public string $line1,
        public string $line2,
        public string $city,
        public string $state,
        public string $zipCode,
        public ?string $phone,
    ) {}

    public function with(
        string|Unchanged $name = Unchanged::Value,
        string|Unchanged $line1 = Unchanged::Value,
        string|Unchanged $line2 = Unchanged::Value,
        string|Unchanged $city = Unchanged::Value,
        string|Unchanged $state = Unchanged::Value,
        string|Unchanged $zipCode = Unchanged::Value,
        string|null|Unchanged $phone = Unchanged::Value,
    ) {
        return new self(
            $name !== Unchanged::Value ? $name : $this->name,
            $line1 !== Unchanged::Value ? $line1 : $this->line1,
            $line2 !== Unchanged::Value ? $line2 : $this->line2,
            $city !== Unchanged::Value ? $city : $this->city,
            $state !== Unchanged::Value ? $state : $this->state,
            $zipCode !== Unchanged::Value ? $zipCode : $this->zipCode,
            $phone !== Unchanged::Value ? $phone : $this->phone,
        );
    }
}

We added the new with() method whose parameters directly correspond to the class properties except that each of them has the special default value Unchanged::Value. This default value allows us to differentiate between the properties we want to change and the ones we want to keep.

All that’s left is to call with() using named arguments, passing only the specific properties we want to change. Et voilà, we made immutable objects a lot more ergonomic to work with.

<?php

$myNewAddress = $myAddress->with(phone: '(555) 555-1234');

// vs.

$myNewAddress = new Address(
    $myAddress->name,
    $myAddress->line1,
    $myAddress->line2,
    $myAddress->city,
    $myAddress->state,
    $myAddress->zipCode,
    '(555) 555-1234'
);

The post Elegant immutable object pattern in PHP appeared first on hbgl.

Laravel shorts - Old inputs are still inputs

hbgl — Thu, 14 Oct 2021 01:44:15 +0000

If you need some inspiration for designing a 500 error page, all you need to do is to find a stylish site that is built on Laravel and navigate to its login page. Then you open up the developer tools and change the name attribute of the email input field from email to email[]. Enter some random credentials and submit.

Disclaimer: Doesn't work on all Laravel sites and the example isn't all that pretty. If you actually did manage to log into someone's account, don't do anything silly.

What happened?

Laravel has a feature where it lets you carry over inputs from an old request to a new request.

As the documentation states:

This feature is particularly useful for re-populating forms after detecting validation errors.

https://laravel.com/docs/8.x/requests#old-input

Here's a quick example:

<form method="POST">
    <h1>Login</h1>
    @csrf
    <input name="email" type="email" value="{{ old('email') }}"><br>
    <input name="password" type="password"><br>
    <input type="submit">
</form>

Now, Laravel also has another useful feature that let's you send array inputs via form fields.

<input name="email[]" type="text" value="foo@bar.com">

class FooController {
    public function handleForm(Request $request) {
        // array:1 [▼
        //   0 => "foo@bar.com"
        // ]
        dd($request->email);
    }
}

Combine the two and somewhere down the pipe you end up with something like htmlspecialchars(['trololo' => 'lololo']) which throws an exception because htmlspecialchars does not take arrays.

Takeaways

Is this a real problem? Probably not, at least not in this example. But I wanted to showcase that old inputs are still inputs and should be treated with zero trust until they are validated. Maybe somewhere in your project you have a bit more complex logic that deals with old inputs. It would not hurt to check that what you think is there is actually there.

The post Laravel shorts – Old inputs are still inputs appeared first on hbgl.

Laravel shorts - Blade component gotcha

hbgl — Sat, 18 Sep 2021 05:16:42 +0000

Update 2021-10-22
An issue has been created on GitHub: https://github.com/laravel/framework/issues/39232

And a fix has been merged:
https://github.com/laravel/framework/pull/39319

Blade components are a great tool for structuring your view templates into small reusable pieces. But while working with them, I stumbled upon a gotcha that I think you should know about. First, let's take a look at this anonymous component:

post-title.blade.php

@props(['post'])
<h1 {{ $attributes }}>{{ $post->title }}</h1>

And here is how it is used in, say, a post list page:

@foreach ($posts as $post)
  <x-post-title :post="$post" />
@endforeach

In this example, the posts are Eloquent models that were previously loaded from the database. Now, if you run this code, you will notice nothing strange in particular. Everything works as intended. However, if you dig a little deeper, you will find some unexpected behavior:

Every time the component is instantiated, the post model is being serialized to JSON.

The cause for this behavior lies in the way the Blade component compiler processes attributes inside the sanitizeComponentAttribute function.

CompilesComponents.php

/**
 * Sanitize the given component attribute value.
 *
 * @param  mixed  $value
 * @return mixed
 */
public static function sanitizeComponentAttribute($value)
{
    return is_string($value) ||
           (is_object($value) && ! $value instanceof ComponentAttributeBag && method_exists($value, '__toString'))
                    ? e($value)
                    : $value;
}

Basically, any object that implements a __toString function will be converted to a string. In the case of an Eloquent model, this will convert it to JSON. Depending on how large your model is and how often the component is used, this will waste a significant amount of resources on constructing JSON strings that are never used.

Solution: Use class based components

Instead of an anonymous component, use a class based component and add your props to the constructor arguments. After you do that, your objects will no longer be converted to strings.

<?php

namespace App\View\Components;

use Illuminate\View\Component;

class PostTitle extends Component
{
    public $post;

    public function __construct($post)
    {
        $this->post = $post;
    }

    public function render()
    {
        return view('components.post-title');
    }
}

The post Laravel shorts – Blade component gotcha appeared first on hbgl.

Backend shorts - Use database transactions

hbgl — Sun, 15 Aug 2021 19:38:41 +0000

Database transactions have many use cases, but in this post we will be only looking at the simplest and most common one: all or nothing semantics aka atomicity.

If you issue multiple related database queries that modify data (INSERT, UPDATE, DELETE), then you should most likely use a transaction. Here is a quick example using Laravel:

function submitOrder(Request $request)
{
    // ....

    $order = new Order($someOrderData);
    $payment = new Payment($somePaymentData);

    DB::transaction(function () use ($order, $payment) {
        $order->save();
        $payment->save();
    }

    // ...
}

Maybe for a better understanding, here is what is going on:

function submitOrder(Request $request)
{
    // ....

    // YOU: Hey DB, I want to save a bunch of data.
    DB::transaction(function () {
        // DB: Ok, tell me everything you want to save.

        // YOU: This order right here.
        $order->save();

        // YOU: And this payment right there.
        $payment->save();

       // YOU: And that's all. Just those two.
    });
    // DB: Alright, order and payment have been saved.

    // ...
}

By wrapping the save() calls in a DB::transaction, we make sure that either all models are saved or none of them. Or said differently, our data is always in a consistent state, which is one of the primary tasks of a backend developer. There will never be an order without a payment.

What could go wrong?

Let's play through the scenario in which we did not use a transaction. Say the order was successfully saved but just when calling $payment->save() the DB goes offline, or the PHP process times out, or it crashes, or the whole machine crashes. You now have an order without a payment, which is probably not something that your application was designed to handle correctly.

The consequences will depend on the exact circumstances, but it is not something that you want to get an angry phone call at 8pm on a Sunday for.

Backend shorts - Validate all your inputs

hbgl — Sat, 14 Aug 2021 20:34:49 +0000

Whenever you are dealing with untrusted data, such as input from a HTTP request, you must validate it thoroughly in the backend of your web application.

function store(Request $request)
{
    $validated = $this->validate($request, [
        'name' => 'required|string|max:255',
        'unit_price' => 'required|int|min:0|max:10000',
        'currency' => 'required|in:USD,EUR',
        'stock' => 'required|int|min:0|max: 1000000',
    ]);

    $product = new Product($validated);
    $product->save();

    return redirect()->route('products.show', ['product' => $product->id]);
}

One of the easiest and safest ways to read request data in Laravel is the $this->validate function inside a controller. It returns an array that contains the valid data (and only the valid data), or it throws a ValidationException if the data is invalid.

You should generally avoid reading data directly from the request object with $request->get('field') or similar methods, because it is very easy to forget to add the necessary validation. Reading data directly from the request object is a code smell.

Takeaways

When it comes to user input, be paranoid.
Always validate in the backend. Frontend validation offers no protection (although it is good for the UX).
Avoid reading unvalidated data directly from the request.

Photo by Filip Olsok from Pexels

Bonus

Sometimes untrusted input data flies under the radar because it is not immediately accessible. A common example for this would be a CSV import where you first need to parse the CSV content in order to get to the fields. Nevertheless, it is vital to validate the parsed data.

public function importStore(Request $request)
{
    // Validate the file metadata.
    $file = $this->validate($request, [
        'file' => 'required|file|mimes:txt,csv|max:5120',
    ])['file'];

    // Load CSV.
    $csv = Reader::createFromPath($file->path());
    $csv->setHeaderOffset(0);
    $header = $csv->getHeader();
    $input = iterator_to_array($csv->getRecords(), false);

    // Set a limit for the maximum number of products per import.
    if (count($input) > 1000) {
        throw ValidationException::withMessages([
            'file' => 'The import is limited to 1000 rows.',
        ]);
    }

    // Do a quick check to see if the header is correct. Although the
    // validation logic further below would also find the error, it
    // would produce multiple error messages for each line in the file.
    if ($header !== ['name', 'unit_price', 'currency', 'stock']) {
        throw ValidationException::withMessages([
            'file' => 'The CSV file does not have the right header.',
        ]);
    }

    // Validate CSV file content.
    $validated = Validator::make($input, [
        '*' => 'required|array',
        '*.name' => 'required|string|max:255',
        '*.unit_price' => 'required|int|min:0|max:10000',
        '*.currency' => 'required|in:USD,EUR',
        '*.stock' => 'required|int|min:0|max: 1000000',
    ])->validate();

    $instant = now();
    foreach ($validated as &$entry) {
        $entry['created_at'] = $instant;
        $entry['updated_at'] = $instant;
    }

    Product::insert($validated);

    return redirect()->route('products.index');
}

Detecting cyclic arrays in PHP – A new approach

hbgl — Mon, 18 Jan 2021 01:50:42 +0000

In Part 1 of this series I explained the problem of detecting cyclic arrays, and why it cannot be done in pure PHP. I also promised to present a solution that is both simple and correct for all possible arrays, which is what this post is about. But before that, let me restate the definition of cyclic arrays from the previous post: An array is cyclic if iterating all of its items and subitems does not terminate.

Here is an example of a cyclic array:

$array = [1, [2, 3]];
$array[1][1] = &$array;

Did you know?

The PHP standard library already performs checks to see if arrays are cyclic in order to avoid infinite recursion. One such example is json_encode (run code):

$array = [1, [2, 3]];
$array[1][1] = &$array;
json_encode($array);
echo json_last_error_msg() . ', ' . json_last_error ();

Recursion detected, 6

This already gives us an easy and correct solution, right (run code)?

function is_cyclic(array &$array) {
    json_encode($array);
    return json_last_error() === JSON_ERROR_RECURSION;
}

Photo by Wendelin Jacober from Pexels

We can do better

While being technically correct, serializing an array to JSON just to find out whether it may or may not be cyclic sounds wasteful. Thankfully, there are other functions that we can use. The one that I think has the least overhead is good ol’ count. It takes a not very well known second parameter, called $mode, that can either have the value COUNT_NORMAL or COUNT_RECURSIVE. This is what happens when we recursively count a cyclic array (run code):

$array = [1, [2, 3]];
$array[1][1] = &$array;
count($array, COUNT_RECURSIVE);

Warning: count(): Recursion detected

The documentation for count says that it will emit an E_WARNING when called with COUNT_RECURSIVE on a cyclic array. With this knowledge, we can set up a custom error handler to trap the warning and check if it was indeed caused by a cyclic array (run code).

function is_cyclic(&$array) {
    $isRecursive = false;
    set_error_handler(function ($errno, $errstr) use (&$isRecursive) {
        $isRecursive = $errno === E_WARNING && mb_stripos($errstr, 'recursion');
    });
    try {
        count($array, COUNT_RECURSIVE);    
    } finally {
        restore_error_handler();
    }
    return $isRecursive;
}

This is already our complete solution. Pretty neat.

Does this really work?

Yep, it does. There is, however, one important thing to consider: PHP is known for changing warnings into exceptions with new versions, and in fact, that is what happened with the upgrade from PHP 7 to PHP 8. This means that in the future the function may need to be slightly adjusted, which I am comfortable with. More importantly, this solution also works for the edge case that was described in Part 1.

Performance is also excellent because the count function barely adds any overhead (see count source code). In my benchmarks, it is considerably faster than a pure PHP solution, on top of being correct.

Does it also work for objects?

Sadly, there is no function like count to efficiently detect recursive objects. There is json_encode and var_dump, but as I already pointed out, they do a lot of unnecessary work. It would be nice if the standard library exposed a function to detect cycles in both arrays and objects.

That being said, lacking a library function to detect cycles in objects is noncritical since it is possible to implement a working algorithm in plain PHP.

Wrapping up

While coming up with my solution I did a lot of looking around in the PHP source code. I wanted to find out how the runtime is able to detect cycles and where this is surfaced in the standard library. It was only through this that I was even able to find out about the count function’s second parameter. I would also encourage everyone to dig a bit deeper and see for yourself how the sausage is made. Even if you have never written a single line of C code, you will probably be able to understand the gist of it and find something interesting.

I wish people were more fearless. There is too many people who don’t understand that it’s just code. It may not be the code that you wrote, but it’s just code. It follows the same rules. It may be in language that you are not particularly familiar with. But it’s still code running on a von Neumann architecture. There is nothing magic here.

Larry Osterman, https://channel9.msdn.com/Shows/Checking-In-with-Erik-Meijer/Checking-In-Larry-Osterman-26-Years-of-Programming-at-Microsoft

Code

You can find tests and benchmarks for the function on GitHub. I also included the broken implementation from Part 1 as well as the json_encode variant for comparison.

The post Detecting cyclic arrays in PHP – A new approach appeared first on hbgl.

Why it is impossible to detect cyclic arrays in pure PHP

hbgl — Mon, 18 Jan 2021 01:48:42 +0000

This post is about a problem that might only be relevant to a small minority of PHP programmers who maybe want to write a custom serializer, or something else that requires recursively iterating an array. But, in trying to solve it, I found some interesting PHP weirdness that I think is worth highlighting.

As an aside :
If you are just looking for a reliable solution for detecting cyclic arrays – and you want to miss out on learning about some cool (but also complicated) PHP quirks – then skip to Part 2 of this series.

First of all, let’s define our terms. For the purpose of this post, we can espouse a very pragmatic definition: An array is cyclic if iterating all of its items and subitems does not terminate. Cyclic arrays are also sometimes called circular or recursive. Let’s look at some examples:

// Contains a reference to itself
$v = [1, 2, 3];
$v[1] = &$v;

// Contains a nested array that contains a reference to $x
$x = [1, [2, 3]];
$x[1][1] = &$x;

// Contains a nested array that contains a reference to an ancestor
$y = [1, [2, [3, 4]]];
$y[1][1][1] = &$y[1];

// Contains a nested array that contains a reference to itself
$z = [1, [2, 3]];
$z[1][1] = &$z[1];

I think you get the idea. All of these arrays contain a cycle at some level.

Attempting a pure PHP solution

Here is a simple algorithm for detecting cyclic arrays: For a given array, iterate all items as well as subitems and mark every array along the way. If at any point you encounter an already marked array, then you found a cycle and the input array is cyclic. This is how one could write it in PHP:

function is_cyclic(array &$array)
{
    $lastKey = array_key_last($array);
    if ($lastKey === null) {
        // Array is empty
        return false;
    }
    static $marker;
    if ($marker === null) {
        $marker = new stdClass();
    }
    if ($array[$lastKey] === $marker) {
        return true;
    }
    $array[] = $marker;
    foreach ($array as &$item) {
        if (is_array($item) && is_cyclic($item)) {
            array_pop($array); // Remove marker
            return true;
        }
    }
    array_pop($array); // Remove marker
    return false;
}

It is worth pointing out that the array argument must be passed by reference; or else we would be appending the marker to a copy and the algorithm would never terminate when given a recursive array as input. Keep this in mind because it will later come back to bite us. But at least for now, this function appears to work just fine (run code).

Breaking the pure PHP solution

Let me preface this section by saying that in PHP you cannot compare arrays for instance identity as you can do with objects. The only way to determine if two variables refer to the same array is to mark the array using one variable and then to check for the mark using the other variable. Thus, if we can find a critical flaw in this procedure, then it is impossible to reliably detect cyclic arrays in pure PHP.

In order to break our function from before, we have to feed it a specially crafted array. There are several ways to do it. The one I am showing you looks very unsuspecting and might actually appear in real code.

function craft_bomb() {
    $array = [1, [2, 3]];
    $array[1][1] = &$array;
    return $array;
}

$bomb = craft_bomb();
var_export(is_cyclic($bomb));

Will this print true or false? The answer is: neither. Instead, this code will never complete until you run out of memory or hit a timeout (run code). How can this be? The array is clearly recursive as var_dump will tell you:

$bomb = craft_bomb();
var_dump($bomb);

array(2) {
  [0]=>
  int(1)
  [1]=>
  array(2) {
    [0]=>
    int(2)
    [1]=>
    *RECURSION*
  }
}

Nothing out of the ordinary here. Just to make sure, let’s dump an array with the same structure, the only difference being that it is constructed in the local scope.

$local = [1, [2, 3]];
$local[1][1] = &$local;
var_dump($local);

array(2) {
  [0]=>
  int(1)
  [1]=>
  array(2) {
    [0]=>
    int(2)
    [1]=>
    *RECURSION*
  }
}

As expected, the dumps are identical (run code), and yet, one array will crash our program and the other one will not. There clearly is more to it than meets the eye.

What is going on inside the array

Photo by cottonbro from Pexels

Let’s do some printf debugging to get an idea about what is actually happening in our program. When we dump the function argument on every call, we get the following output (run code):

Call 1:
array(2) {
  [0]=>
  int(1)
  [1]=>
  array(2) {
    [0]=>
    int(2)
    [1]=>
    _RECURSION_
  }
}

Call 2:
array(2) {
  [0]=>
  int(2)
  [1]=>
  array(2) {
    [0]=>
    int(1)
    [1]=>
    _RECURSION_
  }
}

Call 3:
Same as call 1

Call 4:
Same as call 2
...

These two array dumps keep on alternating forever. Interestingly enough, we would get the exact same output if we were to pass the function argument by value instead of by reference, as I alluded to earlier. It suggests that there is some copying going on, which is strange because one cannot tell by just looking at the function code.

What is perhaps more insightful than the function argument, is how the original input array changes. When we dump it for each function call, we see something quite peculiar (run code):

Call 1:
array(2) {
  [0]=>
  int(1)
  [1]=>
  array(2) {
    [0]=>
    int(2)
    [1]=>
    _RECURSION_
  }
}

Call 2:
array(3) {
  [0]=>
  int(1)
  [1]=>
  &array(2) {
    [0]=>
    int(2)
    [1]=>
    array(2) {
      [0]=>
      int(1)
      [1]=>
      _RECURSION_
    }
  }
  [2]=>
  object(stdClass)#1 (0) {
  }
}

Call 3:
array(3) {
  [0]=>
  int(1)
  [1]=>
  &array(3) {
    [0]=>
    int(2)
    [1]=>
    &array(2) {
      [0]=>
      int(1)
      [1]=>
      array(2) {
        [0]=>
        int(2)
        [1]=>
        _RECURSION_
      }
    }
    [2]=>
    object(stdClass)#1 (0) {
    }
  }
  [2]=>
  object(stdClass)#1 (0) {
  }
}

...

That is quite a mouthful, but, we see that the dump of the array grows one level deeper on every function call. In addition to that, the memory usage of the script steadily increases over time (run code). This leads me to believe that the runtime is lazily materializing the cyclic array in memory as we go down the call chain. It would also explain the mechanism that makes our program run forever.

What we still do not know is why the array is behaving this way and why it is different from the locally constructed array.

Arrays cannot be trusted

Photo by Sharath G. from Pexels

Well, in general, arrays work as one would expect. However, when you perform a value assignment with an array that contains a reference, that predictability goes out the window. “Why?” you might ask. Take a look at this piece of code and think about what output you would expect (run code):

$a = 1;
$b = [&$a];
$c = $b;
$b[0] = 2;
echo "$b[0] $c[0]"; // 2 2

If you said 2 2 then you would be correct. But it might as well have been 2 1 depending on the rest of your program, as this example shows (run code):

$a = 1;
$b = [&$a];
$c = $b;
unset($a); // <--
$b[0] = 2;
echo "$b[0] $c[0]"; // 2 1

There is a section about this exact behavior in chapter 4 of the PHP Language Specification that deals with the subject of value assignments of arrays. To summarize, there are two key takeaways that are relevant for our problem:

Any items in the source array which are aliases (references) get assigned to the destination array either by value or by reference, as defined by a runtime-specific algorithm.

$a = 1;
$b = [&$a]; // $b[0] is an alias for $a
$c = $b;
// The last line is equivalent to either:
// $c = [&$a] or $c = [$a];

The actual copying of the array might get deferred until the array is mutated (copy on write).

$a = 1;
$b = [&$a];
$c = $b;
// The last line might be sort of equivalent to $c = &$b
// until the array is mutated through either $b or $c.

With these two rules we can rationalize why the function is_cyclic behaves differently for an array constructed in the local scope and an array returned by a function.

In the first case, no value assignments are ever performed with the array, which is why the function behaves as one would expect.

In the second case, the array is returned from the function by value and then assigned to a local variable; copying being deferred (rule 2). When we want to mark the array, the runtime will first make a copy which will receive the mark (rule 2). Performing the copy also involves copying the nested array, which is again deferred, and the cycle repeats. Every time we mark an array, a new copy gets made and the initial arrays grows one level deeper.

I know, this may be hard to wrap your head around, so I created a diagram that illustrates the abstract memory model for this scenario. It loosely follows the memory model notation from the PHP Language Specification.

The morale of this story

Try to avoid copying arrays that contain aliases. The behavior is runtime dependent and the behavior might not be what you expect. Quoting from the PHP Language Specification:

For portability, it is generally recommended that programs written in PHP should avoid performing value assignment with a right-hand side that is an array with one or more elements or sub-elements that have an alias relationship.

PHP Language Specification, https://phplang.org/spec/04-basic-concepts.html

Wrapping up

As I said at the beginning, if you are interested in a solution for reliably detecting cyclic arrays – which very simple, I promise – then please check out Part 2 of this series.

You can find all code samples from this post in this GitHub repository.

Related work:

Copy-on-write in the PHP language (2009), Akihiko Tozawa, Michiaki Tatsubori, Tamiya Onodera, Yasuhiko Minamide

The post Why it is impossible to detect cyclic arrays in pure PHP appeared first on hbgl.

Overriding Composer packages at the source level

hbgl — Thu, 22 Oct 2020 01:09:36 +0000

I think we all have been in that situation where we had to extend some library class but there was something preventing us from doing it the normal way. A common example of this would be overriding private methods, or accessing private variables, or dealing with final classes. Another maybe less obvious case is when the class you want to extend is a hardwired dependency for code that you have no control over. In that that case, even if everything were public and overridable you would not be able to inject your subclass.

The scenario

In a previous blog post I wrote about wanting to override the function cleanBindings on the Builder class (source) from the Laravel framework. Extending the class itself is no problem because the function is public and the class is not marked final. The real problem here is that the Builder is a hard-coded dependency multiple levels deep in the framework code.

Now, as a disclaimer, there are other ways to accomplish this task that are a lot more conventional than what I am about to show you. And I generally do not recommend that you use my solution for production code. With that out of the way, let’s look at the mechanism.

Overwriting instead of overriding

PHP being a scripting language, any dependency you install with Composer ends up being nothing more than a bunch of PHP source files on your disk. In theory, instead of overriding a library function by extending a class one could just open the corresponding source file under the vendor directory and edit the code directly.

Of course, this comes with a bunch of problems. For one, the change will only persist until you update or re-install the package. And second, nobody else on your team will receive any of your changes. Furthermore, you have to make sure that your change will not break anyone who depends on that function. This may also includes other packages you installed. But if you can guarantee the latter then it’s just a matter of automating the source code change.

Photo by Arralyn from Pexels

Doing the dirty deed

We have to make sure that our source code changes get applied when installing the package and are not overwritten by updates. Conveniently, composer provides a hook for us in the form of the post-autoload-dump event. The documentation states:

post-autoload-dump: occurs after the autoloader has been dumped, either during install/update, or via the dump-autoload command.

https://getcomposer.org/doc/articles/scripts.md#event-names

According to the composer scripts documentation we can define the hook callback as a static function in an autoloadable class. With our project structure as follows we can define our hook in the composer.json file.

project
|-- scripts
| +-- ComposerScripts.php
|-- src
|-- vendor
|-- composer.json
+-- composer.lock


{
    "autoload": {
        "psr-4": {
            "App\\": "src/",
            "Scripts\\": "scripts/"
        }
    },
    "scripts": {
        "post-autoload-dump": [
            "Scripts\\ComposerScripts::postAutoloadDump"
        ]
    }
}

Let’s quickly check if it works with this ComposerScripts class.

<?php

namespace Scripts;

class ComposerScripts
{
    public static function postAutoloadDump($event)
    {
        echo 'I am gonna use this'.PHP_EOL;
    }
}


$ composer dump-autoload
Generating autoload files
Scripts\ComposerScripts::postAutoloadDump
I am gonna use this
Generated autoload files

Head transplant

Photo by Shuxuan Cao from Pexels

Next we want to replace the function body of cleanBindings in the source file with our own. But how do we do it in a way that is not so fragile as to break down the next time the Laravel devs update the file upstream?

We can make use of the fantastic nikic/PHP-Parser project to parse the PHP code into an AST, replace the function body, and then convert it back into PHP code. Sounds like a lot of work but the parser does all the heavy lifting for us.

use PhpParser\Node;
use PhpParser\NodeFinder;
use PhpParser\ParserFactory;

function replaceCleanBindingsBody(string $srcPath)
{
    $code = file_get_contents($srcPath);

    // Parse the source code.
    $parser = (new ParserFactory())->create(ParserFactory::PREFER_PHP7);
    $stmts = $parser->parse($code);

    // Find the node of the cleanBindings function in the AST.
    $nodeFinder = new NodeFinder();
    $builderClass = $nodeFinder->findFirstInstanceOf($stmts, Node\Stmt\Class_::class);
    $cleanBindingsFunction = $nodeFinder->findFirst($builderClass->stmts, function(Node $node) {
        return $node instanceof Node\Stmt\ClassMethod
            &amp;&amp; $node->name->toString() === 'cleanBindings';
    });

    $newCleanBindingsCode = <<<'PHP'
<?php
function donor() {
    // Up to you
    return [];
}
PHP
    ;

    // Transplant the body of the donor to our patient.
    $donorFunction = $parser->parse($newCleanBindingsCode)[0];
    $cleanBindingsFunction->stmts = $donorFunction->stmts;

    // Dump PHP code.
    $prettyPrinter = new \PhpParser\PrettyPrinter\Standard();
    $newCode = $prettyPrinter->prettyPrintFile($stmts);

    file_put_contents($srcPath, $newCode);
}

What’s left?

In terms of coding, we need to require the composer autoloader file in the ComposerScripts class in order to use the PHP-Parser. I uploaded a full sample project with the things discussed in this post.

Use this power responsibly.

TO THE MAXIMUM EXTEND PERMITTED BY APPLICABLE LAW, THE AUTHOR OF THIS BLOG POST SHALL NOT BE LIABLE FOR ANY DAMAGE TO CODE BASES, OR PRODUCTION SYSTEMS, OR THE TIME LOST DUE TO HOUR-LONG DEBUGGING SESSIONS RESULTING FROM APPLYING THE TECHNIQUES DESCRIBED IN THIS BLOG POST. THE MATERIALS IN THIS BLOG POST COMPRISE THE AUTHOR'S VIEWS AND DO NOT CONSTITUTE PROFESSIONAL ADVICE, you know.

The post Overriding Composer packages at the source level appeared first on hbgl.

Postgres Arrays and PHP -Part 1

hbgl — Sat, 17 Oct 2020 21:50:25 +0000

The other day I found myself designing a database schema for a collection of musicians each of which plays one or more instruments. The result looked something like this.

CREATE TABLE musicians
(
    id bigserial NOT NULL,
    name text NOT NULL,
    PRIMARY KEY (id)
);

CREATE TYPE musical_instrument AS ENUM
    ('guitar', 'piano', 'bass', 'trumpet', 'drums');

CREATE TABLE musician_instruments
(
    musician_id bigint NOT NULL,
    instrument musical_instrument NOT NULL,
    "position" integer NOT NULL,
    PRIMARY KEY (musician_id, instrument)
);

As you can see the table musician_instruments is used to associate musical instruments to a musician. It has an additional column called position because the order of instruments is relevant for the application.

When I squinted real hard I saw that I had in front of me a simple array disguised table. So I thought, why not use the array type that Postgres provides and see where it leads me. A research project if you might.

ALTER TABLE public.musicians
    ADD COLUMN instruments musical_instrument[] NOT NULL;

Using Postgres arrays with PHP

Photo by Aleks Magnusson from Pexels

The application that accesses the database is written in PHP using the Laravel framework which uses PDO as the mechanism to talk to the database. Let’s query the table and see what happens.

This is the musicians table:

id	name	instruments
1	Peter	{guitar,piano}

This is the query:

$musicians = DB::select('SELECT * FROM musicians');
var_dump($musicians);

And this is what we got:

array(1) {
  [0] =>
  class stdClass#728 (3) {
    public $id =>
    int(1)
    public $name =>
    string(5) "Peter"
    public $instruments =>
    string(14) "{guitar,piano}"
  }
}

Photo by Radovan Zierik from Pexels

Meh. For a second there I had my hopes up that PDO would return the instruments as a PHP array all ready to go. Lucky for us, the string looks easy enough to parse.

function maybeParseArrayOutput(string $output) {
    if ($output === '{}') {
        return [];
    }
    return mb_split(',', mb_substr($output, 1, -1));
}
$instruments = maybeParseArrayOutput('{guitar,piano}');
// ['guitar', 'piano']

This works fine for a subset of arrays and if you know that your arrays fall into that subset then you’re good to go. However, leaving aside the issue of nested arrays and arrays of complex types, there are common cases where this simple solution breaks down.

Postgres Array Output Syntax

Let’s take a look at how Postgres formats the array output:

The array output routine will put double quotes around element values if they are empty strings, contain curly braces, delimiter characters, double quotes, backslashes, or white space, or match the word NULL. Double quotes and backslashes embedded in element values will be backslash-escaped.

https://www.postgresql.org/docs/current/arrays.html#ARRAYS-IO

The rules seem pretty straight forward but do we really want to write a parser? It is very easy to overlook some of the edge cases and have a potentially exploitable bug in our application. I leave it as an challenge for the adventurous reader to write a parser that can handle just all possible text[] returned by Postgres.

Fortunately, there is a more pragmatic solution that makes use of the function array_to_json to convert the Postgres array to JSON which is something that PHP can natively parse:

$musicians = DB::select('SELECT id, name, array_to_json(instruments) as instruments FROM musicians');
foreach ($musicians as $musician) {
    $musician->instruments = json_decode($musician->instruments);
}
var_dump($musicians);

array(1) {
  [0] =>
  class stdClass#2032 (3) {
    public $id =>
    int(1)
    public $name =>
    string(5) "Peter"
    public $instruments =>
    array(2) {
      [0] =>
      string(6) "guitar"
      [1] =>
      string(5) "piano"
    }
  }
}

Conversely we can convert a JSON array to text[] when inserting new records.

DB::insert(<<<SQL
    INSERT INTO musicians (name, instruments) VALUES
        ('Mike', ARRAY(SELECT * FROM json_array_elements_text(?))::musical_instrument[]);
SQL
, [
    json_encode(['drums', 'bass']),
]);

So what’s the point?

Photo by László Virág from Pexels

We are already converting our data to and from JSON so why not just store it in a JSON column? Actually, if you want a practical solution that integrates nicely with Laravel then you are better off storing your data as JSON directly. Of course as a trade-off you lose the strong type checking and self-documenting nature of the native Postgres array.

But let’s not give up on the idea just yet. Let’s first find out how we can use Postgres arrays together with the Laravel query builder.

Postgres Arrays with Laravel Query Builder

As you might already expect, selecting with the query builder is no problem.

DB::table('musicians')->select([
    'id',
    'name',
    DB::raw('array_to_json(instruments) as instruments'),
]);

Inserting new rows, on the other hand, is the first major roadblock.

DB::table('musicians')->insert([
    'name' => 'Paula',
    'instruments' => DB::raw('ARRAY(SELECT * FROM json_array_elements_text(?))::musical_instrument[]')
]);

Laravel has no built-in mechanism to fill that ? placeholder. First instinct is to look at the source code of insert and check if where we can somehow extend it. Scrolling to the end of the function we see that it is ultimately the call to cleanBindings which maps our values to bindings.

// From Illuminate\Database\Query\Builder

public function insert(array $values)
{
    // [...]
    return $this->connection->insert(
        $this->grammar->compileInsert($this, $values),
        $this->cleanBindings(Arr::flatten($values, 1))
    );
}

public function cleanBindings(array $bindings)
{
    return array_values(array_filter($bindings, function ($binding) {
        return ! $binding instanceof Expression;
    }));
}

The values from our input array become bindings with expressions filtered out for obvious reason. For our problem we need to do the opposite. We need the expression to provide additional bindings. To do that, we have to first extend the Expression class.

use Illuminate\Database\Query\Expression;

class ParameterizedExpression extends Expression
{
    protected $bindings;

    public function __construct($value, array $bindings)
    {
        parent::__construct($value);
        $this->bindings = $bindings;
    }

    public function getBindings()
    {
        return $this->bindings;
    }
}

Now we can modify cleanBindings to extract the bindings from the parameterized expression.

public function cleanBindings(array $bindings)
{
    $items = [];
    foreach ($bindings as $binding) {
        if ($binding instanceof ParameterizedExpression) {
            foreach ($binding->getBindings() as $b) {
                $items[] = $b;
            }
        } else if (!($binding instanceof Expression)) {
            $items[] = $binding;
        }
    }
    return $items;
}

With this we can use the insert function with arrays like this:

DB::table('musicians')->insert([
    'name' => 'Paula',
    'instruments' => new ParameterizedExpression('ARRAY(SELECT * FROM json_array_elements_text(?))::musical_instrument[]', [json_encode(['trumpet'])]),
]);

It works but we can make it much cleaner if we refactor out all of the boilerplate into a helper function.

class Expr {
    public static function array(array $items, ?string $elementType = null): ParameterizedExpression
    {
        $sql = 'ARRAY(SELECT * FROM json_array_elements_text(?))';
        if ($elementType !== null) {
            $sql .= "::{$elementType}[]";
        }
        return new ParameterizedExpression($sql, [json_encode($items)]);
    }
}

DB::table('musicians')->insert([
    'name' => 'Paula',
    'instruments' => Expr::array(['trumpet'], 'musical_instrument'),
]);

It is still necessary to specify the array type but I can live with that, plus when working with text arrays it can be omitted entirely. One major advantage of this solution is that we can use parameterized expressions not only for insert but also for update and all of the different variants. The big downside is that overriding cleanBindings involves a lot of work since the dependency on the Builder class is hard coded pretty deep down the chain. It’s almost bad enough to warrant some composer shenanigans.

Wrapping up

By now if you are not already convinced that using Postgres arrays with the current state of PHP libraries is a bad idea then you can look forward to Part 2 in which I am going to have a go at using Postgres arrays together with Eloquent Models.

You can find a repository with the code samples from this blog at github.

The post Postgres Arrays and PHP -Part 1 appeared first on hbgl.

Add columns with custom types in Laravel Migrations

hbgl — Wed, 07 Oct 2020 16:30:02 +0000

I was certainly surprised when I found out that there is no built-in method to create table columns with custom types in Laravel migrations. The default list of column types available in Laravel may be more than enough for most applications, but, I think there are some valid use cases for opening the hood and directly working on the engine.

The scenario

Many DBMS offer non-standard column types that you might want to use to unlock some untapped potential. For me this was adding support for PostgreSQL’s built in enumerated types (enums) and arrays. Now, I am not claiming that using these particular types is generally a good idea (especially arrays). As always, you need to evaluate the upsides and downsides for your specific application.

How it’s done

Say, you have an enum type named “foo” and you want to create a new table with a column of this type. You might intuitively reach for the addColumn function as it let’s you specify a type name:

<?php
DB::unprepared("CREATE TYPE foo AS ENUM ('a', 'b', 'c');");

Schema::create('my_table', function (Blueprint $table) {
    $table->bigIncrements('id');
    $table->addColumn('foo', 'my_foo_col');
});

If you ran this migration you would be greeted with the following error message:

Method Illuminate\Database\Schema\Grammars\PostgresGrammar::typeFoo does not exist.

You see, when adding a column of any type to a table there must exist a corresponding method typeYourTypeName in the respective Grammar class (in my case PostgresGrammar). This method is expected to return the underlying type name for your specific DBMS, which is necessary to even out the differences between DBMS vendors. For example an Integer in PostgreSQL will get your an int whereas in MySQL it is integer.

So, in order to get rid of the error we need to somehow make a method named typeFoo available on the PostgresGrammar class. Of course, we could subclass the grammar, add our method, and then inject it using Laravel’s service container, but, that is a lot of work. Luckily for us, the grammar class uses the trait Macroable through which we can add our method to the existing class.

<?php
Grammar::macro('typeFoo', function () {
    return 'foo';
});

DB::unprepared('CREATE TYPE foo AS ENUM ('a', 'b', 'c');');

Schema::create('my_table', function (Blueprint $table) {
    $table->bigIncrements('id');
    $table->addColumn('foo', 'my_foo_col');
});

With only an extra three lines of code our migration runs fine and does what we want.

Make it generic

The practitioners among you might call it a day, check it in, and deploy, which is perfectly fine. But, what if we wanted to add more enum types in the future? Do we really need to add a macro for each one of them? Fortunately not.

We can make use of the fact that the addColumn function takes an optional third parameter appropriately named $parameters which can be used to specify additional options for the column type. With it we can create a single macro that can handle all enum types.

<?php
Grammar::macro('typeRaw', function (Fluent $column) {
    return $column->get('raw_type');
});

DB::unprepared("CREATE TYPE foo AS ENUM ('a', 'b', 'c');");
DB::unprepared("CREATE TYPE bar AS ENUM ('x', 'y', 'z');");

Schema::create('my_table', function (Blueprint $table) {
    $table->bigIncrements('id');
    $table->addColumn('raw', 'my_foo', ['raw_type' => 'foo']);
    $table->addColumn('raw', 'my_bar', ['raw_type' => 'bar']);
    $table->addColumn('raw', 'my_multiple_foos', ['raw_type' => 'foo[]']);
});

To wrap it all up, we can define a macro on the Blueprint class to get rid of the verbose syntax and make it look all nice and clean.

<?php
Blueprint::macro('addColumnRaw', function ($rawType, $name) {
    return $this->addColumn('raw', $name, ['raw_type' => $rawType]);
});

$table->addColumnRaw('foo', 'my_foo');

Check out this github repository for a full working sample and in particular this migration file for the relevant code.

The post Add columns with custom types in Laravel Migrations appeared first on hbgl.