DEV Community: André Laugks

Using nested annotations for key-value pairs in a custom annotation

André Laugks — Sun, 19 Jan 2025 18:44:22 +0000

Introduction

In my article "Using a hashmap in a custom annotation" I explained how to use a HashMap in an annotation using an Enum Constant.

Annotations are allowed as type in an annotation. This means that nested annotations can be used to define key-value pairs in a custom annotation.

List of supported Types in Annotations

Annotations

Two customs annotations are required. The first annotation (e.g. MapItem) containing a key-value pair and the second annotation (e.g. MapItems) containing a list of MapItem annotations.

Custom Annotation @MapItem

The annotation @MapItem represents a single key-value pair.

@Target(ElementType.FIELD)
public @interface MapItem {

    String key();
    String value();
}

Custom Annotation @MapItems

The annotation @MapItems defines a list of MapItem.

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @interface MapItems {

    MapItem[] items();
}

Functional Test

The list of @MapItem annotations are set in the @MapItems annotation.

class ExampleDto {

    @MapItems(items = {
        @MapItem(key = "1", value = "MALE"),
        @MapItem(key = "2", value = "FEMALE"),
        @MapItem(key = "6", value = "DIVERS")
    })
    public String salutation;
}

The MapItemsTest tests the MapItems annotation. The test is done on the salutation field.

To demonstrate how the list of @MapItem can be used, I create a HashMap from @MapItem and compare it to an expected HashMap.

class MapItemsTest {

    @Test
    void testMapItems() throws NoSuchFieldException {

        Field field = ExampleDto.class.getDeclaredField("salutation");
        field.setAccessible(true);

        MapItems annotation = field.getAnnotation(MapItems.class);

        Map<String, String> mappingItems = Arrays
                .stream(annotation.items())
                .collect(
                    Collectors.toMap(
                        MapItem::key,
                        MapItem::value
                    )
        );

        assertEquals(
            new HashMap<>() {{
                put("1", "MALE");
                put("2", "FEMALE");
                put("6", "DIVERS");
            }},
            mappingItems
        );
    }
}

Conclusion

Pro

It is a smart solution and easy to implement.

Contra

If the key-value pairs are to be used in a validator, for example, they must be fetched indirectly.

Full example

https://github.com/alaugks/article-annotation-key-value-pairs

Using a HashMap in a custom Annotation

André Laugks — Tue, 17 Dec 2024 12:43:41 +0000

Introduction

There may be a requirement to define key-value pairs in an annotation in order to perform a mapping based on keys or/and values.

In this case a HashMap would be the first choice. However, annotations do not support the type HashMap. A HashMap can be used with the Enum type, which is supported by annotations.

List of supported Types in Annotations

In my article "Creating a custom Jackson JsonSerializer and JsonDeserializer for mapping values", I create the custom annotation @MappingTable to use key-value pairs for bidirectional mapping in a custom JsonSerializer and JsonDeserializer.

Enum Constant

The Maps enumeration type contains the constant SALUTATION_MAP. The HashMaps are defined in these constants using Map.of(). Alternatively, new HashMap<>() can be used.

import java.util.Map;

public enum Maps {

    SALUTATION_MAP(Map.of("1", "MALE", "2", "FEMALE", "6", "DIVERS"));

    private final Map<String, String> map;

    Maps(Map<String, String> map) {
        this.map = map;
    }

    public Map<String, String> getMap() {
        return this.map;
    }
}

Custom Annotation @MapItems

The enumeration type Maps is set as the type in the annotation.

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @interface MapItems {

    Maps map();
}

Functional Test

The enum constants are referenced in the @MapItems annotation.

class ExampleDto {

    @MapItems(map = Maps.SALUTATION_MAP)
    public String salutation;
}

The MapItemsTest tests the MapItems annotation. The test is done on the salutation field.

class MapItemsTest {

    @Test
    void testEnumConstants() throws NoSuchFieldException {
        Field field = ExampleDto.class.getDeclaredField("salutation");
        field.setAccessible(true);

        MapItems annotation = field.getAnnotation(MapItems.class);

        assertSame(Maps.SALUTATION_MAP.getMap(), annotation.map().getMap());
    }
}

Conclusion

Pro

Simple implementation
Reuse of the HashMap, for example, when validating a property of a payload, validating fields in a DTO or collecting key-value pairs in a response.

Contra

–

Full example

https://github.com/alaugks/article-annotation-key-value-pairs

Problems with webfonts (an article from 1998)

André Laugks — Fri, 12 May 2023 16:27:01 +0000

I wrote a small article about the "Problems with downloadable font files" in early 1998 and it was published on SELFHTML.

At that time, webfonts were also called "dynamic font" or "downloadable font".

SELFHTML was the most popular and comprehensive reference for HTML, CSS, JavaScript and XML in the German-speaking countries. There was also a forum where technical questions could be asked and answered by users.

In 1998, webfonts were totally new and it required a lot of effort to integrate them into a website. Also, there was no high speed internet for private users in Germany at that time. Also special attention was needed to integrate them into your website because of slow download speed.

I wrote this original article in 1998 in German and translated this article now automatically with DeepL into English.
The hyperlinks in my article to TrueDoc (truedoc.com), Netscape (netscape.com) and Microsoft (microsoft.com) are broken because the companies or sites no longer exist. The hyperlinks point to snapshots (late 1998 to early 1999) on archive.org.
The links to the SELFHTML (version from 1998) mirror in my article open in Google Translate (German to English).

Follow the German article translated in English:

Problems with downloadable font files

Based on a posting by:
André Laugks

You can read about the different approaches taken by Microsoft and by Netscape in the section Downloadable Fonts. There are several ways to accommodate the differences between the two browsers.

1. Possibility: Load Active X Control in Internet Explorer

The company TrueDoc provides an ActiveX control with which pfr font files can be displayed in Internet Explorer. In the header of your document, a link to an external JavaScript file and pfr font files according to HTML syntax or according to CSS syntax is included. The external JavaScript file contains a browser query. If your document is loaded in Internet Explorer, an ActivX control is loaded from the TrueDoc server. The external JavaScript file and the ActiveX control are freely available on the TrueDoc server. Detailed information about the ActiveX control, external JavaScript file and copyright can be found on the TrueDoc homepage.

http://www.truedoc.com/webpages/FAQs/activex_faq.html
FAQ about the TrueDoc ActiveX Control

http://www.truedoc.com/webpages/intro/howto_ie.htm
Using pfr font files in IE 4

This method has the advantage that you do not have to burn a eot font file. This simplifies the management of font files within your project. In practice, the Active X Control is used most often.

2.1 Option: Load browser query and font file dynamically

If you decide to use a pfr and eot font file, you can use a browser query to determine whether your page is loaded in Netscape Navigator or Internet Explorer. You can easily and reliably query Internet Explorer 4.0 and above via document.all and Netscape Navigator 4.01 and above via document.layers. Depending on the browser, a JavaScript dynamically writes the CSS syntax into the header of your page.

<html>
<head>
    <script language="JavaScript">
        <!--
        document.writeln('<style type="text/css"><!--')
        document.writeln('@font-face {');
        document.writeln('font-family:Chianti XBd BT;');
        if(document.layers) {
            document.writeln('src:url(http://www.meine.com/fonts/chianti.pfr); }');
        }
        if(document.all) {
            document.writeln('src:url(http://www.meine.com/fonts/chianti.eot); }');
        }
        document.writeln('//--></style>')
        //-->
    </script>
</head>
<body>
    <p style="font-family:Chianti XBd BT">Text in sample font</p>
</body>
</html>

2.2 Load separate CSS file

You also have the option of using a browser query to load a separate CSS file containing the CSS syntax.
You should test this procedure extensively in Netscape Navigator, as it often has problems with the dynamic loading of separate CSS and JavaScript files.

<html>
<head>
    <script language="JavaScript">
        <!--
        if(document.layers) {
            document.writeln('<link rel="stylesheet" type="text/css" href="nn.css">');
        }
        if(document.all) {
            document.writeln('<link rel="stylesheet" type="text/css" href="ie.css">')
        }
        //-->
    </script>
</head>
<body>
    <p style="font-family:Chianti XBd BT">Text in sample font</p>
</body>
</html>

/* separate CSS file for Netscape Navigator - nn.css */

@font-face {
    font-family:Chianti XBd BT;
    src:url(http://www.meine.com/fonts/chianti.pfr);
}

/* separate CSS file for Internet Explorer - ie.css */

@font-face {
    font-family:Chianti XBd BT;
    src:url(http://www.meine.com/fonts/chianti.eot);
}

"Risks and side effects" of downloadable fonts

Due to browser differences and the lack of support for downloadable font files by older browsers or by Opera and due to bandwidths, downloadable font files are unfortunately also fraught with risks and side effects!

Netscape Navigator up to version 4.01, Internet Explorer up to version 4.0 and Opera do not support downloadable font files.
a font file can be up to 150kB in size, this can result in a long download time
the ActiveX control is 69kB in size, this may result in a long download time
ActiveX controls may be disabled in the visitor's Internet Explorer, so the pfr file will not load in Internet Explorer.
JavaScript and CSS might be disabled in the visitor's browser
unloaded font files may mess up the layout or design of the page, the intended effect will not be achieved
the font is only displayed when the font file is completely loaded, before that the alternatively specified font is displayed

Mitigating risks and side effects of downloadable font files

If you do decide to use downloadable font files, there are a few things you should consider:

define alternative fonts: <p style="font-family: "Chianti XBd BT", arial;">text in sample font</p>
for browsers that do not support downloadable fonts, possibly provide an alternative page.
before burning the font file, remove the characters you do not need

Further links to downloadable font files

http://www.truedoc.com/webpages/availpfrs/avail_pfrs.htm
Available PFRs - large selection of ready-burned pfr files

http://www.truedoc.com/webpages/getstart/get_start.htm
Get started - Tutorial on how to include pfr files in Document

http://www.microsoft.com/typography/web/embedding/weft/olddefault.htm
Tutorial on creating eot font files with the tool Microsoft WEFT

http://www.microsoft.com/typography/web/embedding/weft2/default.htm
Tutorial for creating eot font files with the tool Microsoft WEFT 2

http://developer.netscape.com:80/viewsource/goodman_fonts.html
Article by Danny Goodman on the use of downloadable fonts

Create a custom Jackson JsonSerializer und JsonDeserializer for mapping values

André Laugks — Thu, 11 May 2023 11:28:08 +0000

For my series of articles, I also wanted to see how this requirement to mapping values could be implemented with Jackson.

The first paragraph "The requirements and history" from the first article describes the requirements for Emarsys to rewrite the values for the payload.

The required packages

com.fasterxml.jackson.core:jackson-databin
com.fasterxml.jackson.datatype:jackson-datatype-jsr310

See the pom.xml in the example for the latest versions.

Minimal structure of a custom JsonSerializer and JsonDeserializer

To solve the requirements to map the values for Emarsys, a custom JsonSerializer and JsonDeserializer is needed. I call these MappingValueSerializer and MappingValueDeserializer.

Below is the minimal structure of a custom MappingValueSerializer and MappingValueDeserializer:

public class MappingValueSerializer extends JsonSerializer<String> {
    @Override
    public void serialize(String value, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
        jsonGenerator.writeString("serialized: " + value);
    }
}

public class MappingValueDeserializer extends JsonDeserializer<String> {
    @Override
    public String deserialize(JsonParser jsonParser, DeserializationContext deserializationContext) throws IOException  {
        String value = jsonParser.getText();
        return "deserialized: " + value;
    }
}

In ContactDto, the fields salutation and marketingInformation for which values have to be rewritten are defined.

Fields/Direction	serialize (T -> String)	deserialize (String -> T)
salutation	"FEMALE" -> "2"	"2" -> "FEMALE"
marketingInformation	true -> "1"	"1" -> true

For the serialize process it is the FieldValueID (String) and for the deserialize process the type String for salutation and the type Boolean for marketingInformation.

So if you want to do the mapping, you would need a JsonSerializer to write the FieldValueID (String) for salutation and marketingInformation and a JsonDeserializer to set the value for stalutation (String) and marketingInformation (Boolean).

Custom Type

However, I only want to have a JsonDeserializer that can process String, Boolean and in the future other types. For this purpose, I create my own type MappingValue<>. Most importantly, I can transport all types with this custom generics.

package com.article.jackson.serializer;

public class MappingValue<T> {
    T value;

    public MappingValue(T value) {
        this.value = value;
    }

    public T getValue() {
        return this.value;
    }
}

ContactDto

First of all the complete ContactDto with all fields and annotations. I will explain the individual annotations below.

package com.article.jackson.dto;

import java.time.LocalDate;

import com.article.jackson.annotation.MappingTable;
import com.article.jackson.serializer.MappingValue;
import com.article.jackson.serializer.MappingValueDeserializer;
import com.article.jackson.serializer.MappingValueSerializer;
import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.JsonFormat;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import com.fasterxml.jackson.datatype.jsr310.deser.LocalDateDeserializer;
import com.fasterxml.jackson.datatype.jsr310.ser.LocalDateSerializer;

@JsonIgnoreProperties(ignoreUnknown = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonAutoDetect(
    fieldVisibility = JsonAutoDetect.Visibility.ANY,
    getterVisibility = JsonAutoDetect.Visibility.NONE,
    setterVisibility = JsonAutoDetect.Visibility.NONE,
    isGetterVisibility = JsonAutoDetect.Visibility.NONE
)
public class ContactDto {

    @JsonProperty("1")
    private String firstname;

    @JsonProperty("2")
    private String lastname;

    @JsonProperty("3")
    private String email;

    @JsonProperty("4")
    @JsonFormat(pattern = "yyyy-MM-dd")
    @JsonSerialize(using = LocalDateSerializer.class)
    @JsonDeserialize(using = LocalDateDeserializer.class)
    private LocalDate birthday;

    @JsonProperty("46")
    @MappingTable(map = Maps.SALUTATION)
    @JsonSerialize(using = MappingValueSerializer.class)
    @JsonDeserialize(using = MappingValueDeserializer.class)
    private MappingValue<String> salutation;

    @JsonProperty("100674")
    @MappingTable(map = Maps.MARKETING_INFORMATION)
    @JsonSerialize(using = MappingValueSerializer.class)
    @JsonDeserialize(using = MappingValueDeserializer.class)
    private MappingValue<Boolean> marketingInformation;

    // other getter and setter

    public String getSalutation() {
        return salutation.getValue();
    }

    public void setSalutation(String salutation) {
        this.salutation = new MappingValue<>(salutation);
    }

    public Boolean getMarketingInformation() {
        return this.isMarketingInformation();
    }

    public Boolean isMarketingInformation() {
        return marketingInformation.getValue();
    }

    public void setMarketingInformation(Boolean marketingInformation) {
        this.marketingInformation = new MappingValue<>(marketingInformation);
    }
}

Custom Annotation @MappingTable and Enum Constants for the MappingTable

The MappingTable with the FieldValueIDs for salutation and marketingInformation must be available in the MappingValueSerializer and MappingValueDeserializer.

Annotation @MappingTable

For this I create a custom annotation @MappingTable that will be read in the MappingValueSerializer and MappingValueDeserializer.

package com.article.jackson.annotation;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import com.article.jackson.dto.Maps;

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @interface MappingTable {
    Maps map();
}

Enum Constants

package com.article.jackson.dto;

import java.util.Map;

public enum Maps {

    SALUTATION(Map.of("1", "MALE", "2", "FEMALE", "6", "DIVERS")),

    MARKETING_INFORMATION(Map.of("1", true, "2", false));

    private final Map<String, Object> map;

    Maps(Map<String, Object> map) {
        this.map = map;
    }

    public Map<String, Object> getMap() {
        return this.map;
    }
}

Field definitions

The enum constants Maps.SALUTATION and Maps.MARKETING_INFORMATION are referenced in the @MappingTable annotation. The HashMaps are used in the JsonSerializer and JsonDeserializer for bi-directional mapping.

@MappingTable(map = Maps.SALUTATION)
private MappingValue<String> salutation;

@MappingTable(map = Maps.MARKETING_INFORMATION)
private MappingValue<Boolean> marketingInformation;

Reading and writing on the fields

The ObjectManager of Jackson writes and reads on the mutators (setter) and accessor (getter, isser) by default.

For the mutator and accessor of salutation and marketingInformation, however, I would like to define the type String or Boolean.

You can use an annotation to instruct Jackson to read and write only on the fields, so we can use the custom type MappingValue<> internally. The reading and writing process thus takes place on the fields and we can define String and Boolean for the mutator and accessor of salutation and marketingInformation.

@JsonAutoDetect(
    fieldVisibility = JsonAutoDetect.Visibility.ANY,
    getterVisibility = JsonAutoDetect.Visibility.NONE,
    setterVisibility = JsonAutoDetect.Visibility.NONE,
    isGetterVisibility = JsonAutoDetect.Visibility.NONE
)

FieldIDs

The FieldIDs can be defined very easy with @JsonProperty.

@JsonProperty("123")

Define custom JsonSerializer and JsonDeserializer

The custom JsonSerializer (MappingValueSerializer) and JsonDeserializer (MappingValueDeserializer) can be defined with @JsonSerialize and @JsonDeserialize on the field.

@JsonSerialize(using = MappingValueSerializer.class)
@JsonDeserialize(using = MappingValueDeserializer.class)

Skip null values

Fields with null as value should not be serialized. This is because the fields that are sent are also updated. The annotation @JsonInclude can be used for this.

@JsonInclude(JsonInclude.Include.NON_NULL)

Ignore unknown properties

Emarsys always returns all fields for a contact in the response. I want only the fields defined in the ContactDto to be mapped so that no exceptions are thrown. The annotation @JsonIgnoreProperties can be used for this:

@JsonIgnoreProperties(ignoreUnknown = true)

MappingValueSerializer and MappingValueDeserializer

In order for the MappingTable, which is defined at the field, to be read, the interface ContextualSerializer must be implemented for the MappingValueSerializer and the interface ContextualDeserializer for the MappingValueDeserializer.

With createContextual(), access to the property is possible and via BeanProperty the annotation can be fetched and the MappingTable can be read out.

MappingValueSerializer

In the MappingValueSerializer, for example, for salutation "FEMALE" is mapped to "2" and marketingInformation true to "1", which is why the FieldValueID is written with jsonGenerator.writeString().

package com.article.jackson.serializer;

import java.io.IOException;
import java.util.Map;

import com.article.jackson.annotation.MappingTableMapReader;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.BeanProperty;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.ser.ContextualSerializer;

public class MappingValueSerializer extends JsonSerializer<MappingValue<?>> implements ContextualSerializer {

    private final Map<String, Object> map;

    public MappingValueSerializer() {
        this(null);
    }

    public MappingValueSerializer(Map<String, Object> map) {
        this.map = map;
    }

    @Override
    public void serialize(MappingValue<?> field, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
        String fieldValueId = this.map.entrySet()
                .stream()
                .filter(e -> e.getValue().equals(field.getValue()))
                .map(Map.Entry::getKey)
                .findFirst()
                .orElse(null);

        jsonGenerator.writeString(fieldValueId);
    }

    @Override
    public JsonSerializer<?> createContextual(SerializerProvider prov, BeanProperty property) {
        return new MappingValueSerializer(
                new MappingTableMapReader(property).getMap()
        );
    }
}

MappingValueDeserializer

In the MappingValueDeserializer the mapping takes place backwards. Here the FieldValueID for salutation and marketingInformation must be mapped accordingly. For salutation "2" to "FEMALE" (String) and for marketingInformation "1" to true (Boolean).

package com.article.jackson.serializer;

import java.io.IOException;
import java.util.Map;

import com.article.jackson.annotation.MappingTableMapReader;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.BeanProperty;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.deser.ContextualDeserializer;

public class MappingValueDeserializer extends JsonDeserializer<MappingValue<?>> implements ContextualDeserializer {

    private final Map<String, Object> map;

    public MappingValueDeserializer() {
        this(null);
    }

    public MappingValueDeserializer(Map<String, Object> map) {
        this.map = map;
    }

    @Override
    public MappingValue<?> deserialize(JsonParser jsonParser, DeserializationContext deserializationContext) throws IOException {
        String fieldValue = jsonParser.getText();

        return new MappingValue<>(this.map.entrySet()
                .stream()
                .filter(e -> e.getKey().equals(fieldValue))
                .map(Map.Entry::getValue)
                .findFirst()
                .orElse(null));
    }

    @Override
    public JsonDeserializer<?> createContextual(DeserializationContext ctxt, BeanProperty property) {
        return new MappingValueDeserializer(
                new MappingTableMapReader(property).getMap()
        );
    }
}

MappingTableMapReader

In the MappingTableMapReader class, the enum constant is retrieved from the annotation and is available in the JsonSerializer and JsonDeserializer.

ackage com.article.jackson.annotation;

import java.util.Map;

import com.article.jackson.exception.MappingTableRuntimeException;
import com.fasterxml.jackson.databind.BeanProperty;

public class MappingTableMapReader {

    private final BeanProperty property;

    public MappingTableMapReader(BeanProperty property) {
        this.property = property;
    }

    public Map<String, Object> getMap() {

        MappingTable annotation = property.getAnnotation(MappingTable.class);

        if (annotation == null) {
            throw new MappingTableRuntimeException(
                String.format(
                    "Annotation @MappingTable not set at property %s",
                    this.property.getMember().getFullName()
                )
            );
        }

        return annotation.map().getMap();
    }
}

Functional Test

To check the implementation, we still need a test. To compare the JSON, I use assertThatJson() from the package json-unit-assertj.

package com.article.jackson.serializer;

import java.io.IOException;
import java.time.LocalDate;

import com.article.jackson.dto.ContactDto;
import com.article.jackson.exception.MappingTableRuntimeException;
import com.article.jackson.fixtures.ContactDtoAnnotationNotSet;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import net.javacrumbs.jsonunit.core.Option;
import org.junit.jupiter.api.Test;

import static net.javacrumbs.jsonunit.assertj.JsonAssertions.assertThatJson;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;

class MappingTableSerializerDeserializerTest {

    String emarsysPayload = """
            {
                "1": "Jane",
                "2": "Doe",
                "3": "jane.doe@example.com",
                "4": "1989-11-09",
                "46": "2",
                "100674": "1"
            }
            """;

    @Test
    void serialize() throws IOException {
        ContactDto contact = new ContactDto();
        contact.setSalutation("FEMALE");
        contact.setFirstname("Jane");
        contact.setLastname("Doe");
        contact.setEmail("jane.doe@example.com");
        contact.setBirthday(LocalDate.of(1989, 11, 9));
        contact.setMarketingInformation(true);
        String json = new ObjectMapper().writeValueAsString(contact);
        assertThatJson(this.emarsysPayload)
                .when(Option.IGNORING_ARRAY_ORDER)
                .isEqualTo(json);
    }

    @Test
    void deserialize() throws IOException {
        ContactDto contact = new ObjectMapper().readValue(this.emarsysPayload, ContactDto.class);
        assertEquals("FEMALE", contact.getSalutation());
        assertEquals("Jane", contact.getFirstname());
        assertEquals("Doe", contact.getLastname());
        assertEquals("jane.doe@example.com", contact.getEmail());
        assertEquals(LocalDate.of(1989, 11, 9), contact.getBirthday());
        assertTrue(contact.isMarketingInformation());
    }
}

Full example on GitHub

https://github.com/alaugks/article-jackson-serializer

Updates

Change GitHub Repository URL (Sep 6th 2024)
Replace JSON with HashMaps (Dec 19th 2024)

Create a custom JMS Serializer handler for mapping values

André Laugks — Mon, 24 Apr 2023 18:27:31 +0000

In the context of my article Create a custom Symfony Normalizer for mapping values, I also wanted to see if a custom normaliser could be implemented for the JMS Serializer, which I also use in many projects. For the JMS Serializer this is a handler.

Create the handler

For the custom handler, the interface SubscribingHandlerInterface must be implemented. In the method getSubscribingMethods() the methods for the direction and formats are defined.

<?php declare(strict_types=1);

namespace App\Handler;

use JMS\Serializer\GraphNavigatorInterface;
use JMS\Serializer\Handler\SubscribingHandlerInterface;
use JMS\Serializer\SerializationContext;
use JMS\Serializer\Visitor\DeserializationVisitorInterface;
use JMS\Serializer\Visitor\SerializationVisitorInterface;

class MappingTableHandler implements SubscribingHandlerInterface
{
    public const HANDLER_TYPE = 'MappingTable';

    public static function getSubscribingMethods(): array
    {
        $methods = [];

        foreach (['json', 'xml'] as $format) {
            $methods[] = [
                'type' => self::HANDLER_TYPE,
                'format' => $format,
                'direction' => GraphNavigatorInterface::DIRECTION_SERIALIZATION,
                'method' => 'serialize',
            ];

            $methods[] = [
                'type' => self::HANDLER_TYPE,
                'direction' => GraphNavigatorInterface::DIRECTION_DESERIALIZATION,
                'format' => $format,
                'method' => 'deserialize',
            ];
        }

        return $methods;
    }

    public function normalize(
        SerializationVisitorInterface $visitor,
        string|bool|null $value,
        array $type,
        SerializationContext $context
    ): ?string
    {
        $mappingTable = $this->getMappingTable($type);

        foreach ($mappingTable as $mKey => $mValue) {
            if ($value === $mValue) {
                return (string)$mKey; // Force string
            }
        }
        return null;
    }

    public function denormalize(
        DeserializationVisitorInterface $visitor,
        $value,
        array $type
    ): mixed
    {
        $mappingTable = $this->getMappingTable($type);

        foreach ($mappingTable as $mKey => $mValue) {
            if ((string)$value === (string)$mKey) {
                return $mValue;
            }
        }
        return null;
    }

    private function getMappingTable(array $type): array
    {
        $mappingTable = [];

        if (!isset($type['params'][0])) {
            throw new \InvalidArgumentException('mapping_table param not defined');
        }

        if ($array = json_decode($type['params'][0], true)) {
            $mappingTable = $array;
        }

        return $mappingTable;
    }
}

Define the FieldID and MappingTable

The FieldID is defined with the attribute #[SerializedName].

The #[Type] attribute is used to define the MappingTable on the $salutation and $marketingInformation properties.

The MappingTables must be noted as a string.

Here is an example with JSON as MappingTable.

<?php declare(strict_types=1);

namespace App\Dto;

use JMS\Serializer\Annotation\SerializedName;
use JMS\Serializer\Annotation\Type;

class ContactDto
{
    #[SerializedName('1')]
    #[Type('string')]
    private ?string $firstname = null;

    #[SerializedName('2')]
    #[Type('string')]
    private ?string $lastname = null;

    #[SerializedName('3')]
    #[Type('string')]
    private ?string $email = null;

    #[SerializedName('4')]
    #[Type("DateTime<'Y-m-d'>")]
    private ?\DateTimeInterface $birthdate = null;

    #[SerializedName('46')]
    #[Type("MappingTable<'{\"1\": \"MALE\", \"2\": \"FEMALE\", \"6\": \"DIVERS\"}'>")]
    private ?string $salutation = null;

    #[SerializedName('100674')]
    #[Type("MappingTable<'{\"1\": true, \"2\": false}'>")]
    private ?bool $marketingInformation = null;

    /* getter and setter */
}

I find the notation of masked JSON cumbersome and difficult to maintain.

Therefore, I have also implemented the possibility of notating a constant as a string in my implementation. It is still a string but easier to maintain.

In Symfony Forms, for example, we can use these constants.

I wrote a UnitTest that checks if the constants are available. Despite a very good IDE, it can happen that the string in Attribute #[Type] is not renamed when the ContactDto or Constants are renamed.

<?php declare(strict_types=1);

namespace App\Dto;

use JMS\Serializer\Annotation\SerializedName;
use JMS\Serializer\Annotation\Type;

class ContactDto
{
    /* other properties */

    public const SALUTATION  = ['1' => 'MALE', '2' => 'FEMALE', '3' => 'DIVERS'];

    public const MARKETING_INFORMATION  = ['1' => true, '2' => false];

    #[SerializedName('46')]
    #[Type("MappingTable<'App\Dto\ContactDto::SALUTATION'>")]
    private ?string $salutation = null;

    #[SerializedName('100674')]
    #[Type("MappingTable<'App\Dto\ContactDto::MARKETING_INFORMATION'>")]
    private ?bool $marketingInformation = null;

    /* getter and setter */
}

Register the handler

Symfony JMSSerializerBundle

If you are using Symfony and the JMSSerializerBundle, the handler still needs to be registered if you are not using the default services.yaml configuration.

services:
    app.handler.mapping_handler:
        class: 'App\Handler\MappingTableHandler'
        tags:
            - { name: 'jms_serializer.handler', type: 'MappingTable', format: 'json' }

JMS Serializer standalone

If you use the JMS Serializer as a standalone library, you must register the handler as follows:

$serializer = SerializerBuilder::create()
    ->configureHandlers(function(HandlerRegistry $registry) {
        $registry->registerSubscribingHandler(
            new MappingTableHandler()
        );
    })
    ->build();

Normalize and denormalize

With the Serializer you can normalize (toArray()), denormalize (fromArray()), serialize (serialize()) and deserialize (deserialize())

I use toArray() and fromArray() because I need an array for the API client:

<?php declare(strict_types = 1);

use JMS\Serializer\Serializer;

private $serializer Serializer

$contactDto = new ContactDto();
$contactDto->setSalutation('FEMALE');
$contactDto->setFirstname('Jane');
$contactDto->setLastname('Doe');
$contactDto->setEmail('jane.doe@example.com');
$contactDto->setBirthdate(new \DateTime('1989-11-09'));
$contactDto->setMarketingInformation(true);

// Normalize
$fields = $this->serializer->toArray($contactDto);
/*
    Array
    (
        [1] => Jane
        [2] => Doe
        [3] => jane.doe@example.com
        [4] => 1989-11-09
        [46] => FEMALE
        [100674] => true
    )
*/

// Denormalize
$contactDto = $this->serializer->fromArray($fields, ContactDto::class);
/*
    App\Dto\ContactDto Object
    (
        [firstname:App\Dto\ContactDto:private] => Jane
        [lastname:App\Dto\ContactDto:private] => Doe
        [email:App\Dto\ContactDto:private] => jane.doe@example.com
        [birthdate:App\Dto\ContactDto:private] => DateTime Object
            (
                [date] => 1989-11-09 15:23:49.000000
                [timezone_type] => 3
                [timezone] => UTC
            )

        [salutation:App\Dto\ContactDto:private] => FEMALE
        [marketingInformation:App\Dto\ContactDto:private] => 1
    )
*/

Links

Full JMS Serializer handler on github

Updates

Series name defined (May 5th 2023)
Update series name (May 8th 2023)
Fix broken links (Dez 30th 2023)
Change GitHub Repository URL (Sep 4th 2024)

Create a custom Symfony Normalizer and Denormalizer for mapping values

André Laugks — Mon, 03 Apr 2023 16:33:06 +0000

The requirements and history

The task was to integrate a CRM (Emarsys) into the e-commerce platform.

The CRM provides predefined system fields and field types. Custom fields can also be created. The label for the field and the field name can be assigned by the user. The FieldID for the field and the FieldValueID for the choices are assigned by the CRM when the field is created. The field names are not relevant when interacting with the Emarsys API.

Here are some examples:

Label	Fieldname	FieldID	Type: FieldValue => FieldValueID
Salutation	salutation	46	Single-choice: Male => 1 Female => 2 Divers => 6
Firstname	firstName	1	Short text
Lastname	lastName	2	Short text
Email	email	3	Long text
Birthdate	birthday	4	Date: YYYY-MM-DD
Marketing Information *	marketing_information	100674	Single-choice: Yes* => 1 No* => 2

*custom

For a creation of a contact in CRM, with this data

Salutation: Female
Name: Jane Doe
Email: jane.doe@example.com
Birthday: 1989-11-09
Marketing Information: Yes

the payload must be:

{
    "1": "Jane",
    "2": "Doe",
    "3": "john.doe@example.com",
    "4": "1989-11-09",
    "46": "2",
    "100674": "1"
}

The response body, for example, when searching for a contact, has the same structure. However, all the fields of the contact are provided.

Built-in mapping in the API-Client

From the beginning, the snowcap/emarsys package has been used as the API client. This package is good, extensible and provides a simple mapping (Fieldname <-> FieldId). The system fields are already stored in the package.

The built-in mapping is sufficient if you have only a few custom fields. However, if you have 30 custom fields and multiple accounts in use, it becomes difficult to maintain and manage.

Use a custom Attribute

So it would be better to configure the FieldID and the mapping to the FieldValueID directly on the properties. So I implemented a custom attribute and a simple PropertyReader and PropertyWriter.

I have taken the solution with the custom attribute out of the project and made it available on github.

Using the Symfony serializer component and creating a custom normalizer

I wanted to see if it was possible to replace the custom attribute with features provided by the Symfony serializer component.

FieldIDs with attribute #[SerializedName]

First the FieldIDs. This is easily done with the #[SerializedName] attribute.

<?php

namespace App\Dto;

use Symfony\Component\Serializer\Annotation\SerializedName;
use Symfony\Component\Serializer\Normalizer\DateTimeNormalizer;

class ContactDto
{
    #[SerializedName("1")]
    private ?string $firstname = null;

    #[SerializedName("2")]
    private ?string $lastname = null;

    #[SerializedName("3")]
    private ?string $email = null;

    #[Context([DateTimeNormalizer::FORMAT_KEY => 'Y-m-d'])]
    #[SerializedName("4")]
    private ?\DateTimeInterface $birthdate = null;

    #[SerializedName("46")]
    private ?string $salutation = null;

    #[SerializedName("100674")]
    private ?boolean $marketingInformation = null;

    /* getter and setter */
}

Normalize/Denormalize and Serialize/Deserialize

With the Serializer you can normalize, denormalize, serialize and deserialize, encode und decode. With serialize(), the normalize() and encode() are called, and with a deserialize(), the denormalize() and decode() are called.

I use normalize() and denormalize() because I need an array for the API client:

<?php

use Symfony\Component\Serializer\Serializer;

$contactDto = new ContactDto();
$contactDto->setSalutation('FEMALE');
$contactDto->setFirstname('Jane');
$contactDto->setLastname('Doe');
$contactDto->setEmail('jane.doe@example.com');
$contactDto->setBirthdate(new \DateTimeImmutable('1989-11-09'));
$contactDto->setMarketingInformation(true);

// Normalize
$fields = $this->serializer->normalize($contactDto);
/*
    Array
    (
        [1] => Jane
        [2] => Doe
        [3] => jane.doe@example.com
        [4] => 1989-11-09
        [46] => FEMALE
        [100674] => true
    )
*/

// Denormalize
$contactDto = $this->serializer->denormalize($fields, ContactDto::class);
/*
    App\Dto\ContactDto Object
    (
        [firstname:App\Dto\ContactDto:private] => Jane
        [lastname:App\Dto\ContactDto:private] => Doe
        [email:App\Dto\ContactDto:private] => jane.doe@example.com
        [birthdate:App\Dto\ContactDto:private] => DateTimeImmutable Object
            (
                [date] => 1989-11-09 13:54:11.000000
                [timezone_type] => 3
                [timezone] => UTC
            )

        [salutation:App\Dto\ContactDto:private] => FEMALE
        [marketingInformation:App\Dto\ContactDto:private] => 1
    )
*/

This cannot yet be sent to CRM via the API because CRM cannot handle the FieldValue FEMALE for $salutation and true for $marketingInformation.

For this I create a custom normalizer and denormalizer.

⚠️ Note: #[SerializedName] with a number as name in Symfony 6.2

Denormalize FieldIDs does not work in Symfony 6.2. This has to do with an array_merge() when denormalizing in the symfony/serializer package in version 6.2. I have created a pull request for a fix. The pull request has already been reviewed. I guess the fix will be merged soon.

✅ This bug is fixed in version 6.3.5.

Create a custom Normalizer

The Symfony serializer component provides several built-in normalizers for transforming data. For example, there is the DateTimeNormalizer to transform a DateTime object into a date format and a date format into a DateTime object.

For my case, I need a normalizer that transforms FEMALE into FieldValueID 2 for $salutation and a denormalizer that transforms FieldValueID 2 into FEMALE.

To do this, I create the MappingTableNormalizer (Normalizer and Denormalizer), which implements the NormalizerInterface and DenormalizerInterface interfaces.

The serializer calls the supportsNormalization() and supportsDenormalization() functions of all registered normalizers and denormalizers to determine which normalizer or denormalizer to use to transform an object.

A custom type must be created to ensure that the MappingTableNormalizer is applied to the $salutation and $marketingInformation properties.

I have created StringValue ($salutation) and BooleanValue ($marketingInformation). The supportsNormalization() and supportsDenormalization() methods check whether the MappingTableNormalizer is responsible for the object.

Here is the full implementation of the MappingTableNormalizer:

<?php

namespace App\Normalizer;

use App\Normalizer\Value\BooleanValue;
use App\Normalizer\Value\StringValue;
use App\Normalizer\Value\ValueInterface;
use Symfony\Component\Serializer\Normalizer\DenormalizerInterface;
use Symfony\Component\Serializer\Normalizer\NormalizerInterface;

class MappingTableNormalizer implements NormalizerInterface, DenormalizerInterface
{
    public const TABLE = 'mapping_table';

    private const SUPPORTED_TYPES = [
        StringValue::class,
        BooleanValue::class
    ];

    public function normalize(mixed $object, string $format = null, array $context = []): ?string
    {
        $mappingTable = $this->getMappingTable($context);

        $key = array_search($object->getValue(), $mappingTable);
        if($key) {
            return (string)$key; // Force string
        }

        return null;
    }

    public function supportsNormalization(mixed $data, string $format = null): bool
    {
        return $data instanceof ValueInterface;
    }

    public function denormalize($data, $type, $format = null, array $context = array()): mixed
    {
        $mappingTable = $this->getMappingTable($context);

        foreach ($mappingTable as $key => $value) {
            if ((string)$key === $data) {
                return new $type($value);
            }
        }

        return new $type(null);
    }

    public function supportsDenormalization($data, $type, $format = null): bool
    {
        return in_array($type, self::SUPPORTED_TYPES);
    }

    private function getMappingTable(array $context): array
    {
        if (!isset($context[self::TABLE]) || !is_array($context[self::TABLE])) {
            throw new \InvalidArgumentException('mapping_table not defined');
        }

        return $context[self::TABLE];
    }
}

Attribute #[Context] for the MappingTable

The #[Context] attribute is used to define the MappingTable on the $salutation and $marketingInformation properties.

<?php

namespace App\Dto;

use App\Normalizer\MappingTableNormalizer;
use App\Normalizer\Value\BooleanValue;
use App\Normalizer\Value\StringValue;
use Symfony\Component\Serializer\Annotation\Context;
use Symfony\Component\Serializer\Annotation\SerializedName;

class ContactDto
{
    // Other properties

    #[Context([MappingTableNormalizer::TABLE => ['1' => 'MALE', '2' => 'FEMALE', '6' => 'DIVERS']])]
    #[SerializedName("46")]
    private ?StringValue $salutation = null;

    #[Context([MappingTableNormalizer::TABLE => ['1' => true, '2' => false]])]
    #[SerializedName("100674")]
    private ?BooleanValue $marketingInformation = null;

    // Other getter and setter

    public function getSalutation(): ?StringValue
    {
        return $this->salutation;
    }

    public function setSalutation(?StringValue $salutation): void
    {
        $this->salutation = $salutation;
    }

    public function getMarketingInformation(): ?BooleanValue
    {
        return $this->marketingInformation;
    }

    public function setMarketingInformation(?BooleanValue $marketingInformation): void
    {
        $this->marketingInformation = $marketingInformation;
    }
}

Register the custom normalizer

The normalizer still needs to be registered if you are not using the default services.yaml configuration.

services:
  serializer.normalizer.mapping_table_normalizer:
    class: 'App\Normalizer\MappingTableNormalizer'
    tags: ['serializer.normalizer']

Skip null values

Then there should be no normalization of properties that are null. These properties will also not be included in the normalized array. Fields that are not in the payload will not be updated in CRM.

framework:
  serializer:
    default_context: '%serializer_default_context%'

parameters:
  serializer_default_context:
    skip_null_values: true

Normalize the ContactDto to an array

<?php

use Snowcap\Emarsys\Client;
use Symfony\Component\Serializer\Serializer;

private Client $client;
private Serializer $serializer;

$contactDto = new ContactDto();
$contactDto->setSalutation(new StringValue('FEMALE'));
$contactDto->setFirstname('Jane');
$contactDto->setLastname('Doe');
$contactDto->setEmail('jane.doe@example.com');
$contactDto->setBirthdate(new \DateTimeImmutable('1989-11-09'));
$contactDto->setMarketingInformation(new BooleanValue(true));

// Normalize ContactDto
$fields = $this->serializer->normalize($contactDto);
/*
    Array
    (
        [1] => Jane
        [2] => Doe
        [3] => jane.doe@example.com
        [4] => 1989-11-09
        [46] => 2
        [100674] => 1
    )
*/

// Create Contact (API-Request)
$this->client->createContact($fields)

Denormalize the array to a ContactDto

<?php

use Snowcap\Emarsys\Client;
use Symfony\Component\Serializer\Serializer;

private Client $client;
private Serializer $serializer;

// Get Contact (API-Request)
$response = $this->client->getContact(['3' => 'jane.doe@example.com']);
$fields = $response->getData()
/*
    Array
    (
        [1] => Jane
        [2] => Doe
        [3] => jane.doe@example.com
        [4] => 1989-11-09
        // other fields
        [46] => 2
        // other fields
        [100674] => 1
        // other fields
    )
*/

// Denormalize Array
$conatctDto = $this->serializer->denormalize($fields, ContactDto::class);
/*
    App\Dto\ContactDto Object
    (
        [firstname:App\Dto\ContactDto:private] => Jane
        [lastname:App\Dto\ContactDto:private] => Doe
        [email:App\Dto\ContactDto:private] => jane.doe@example.com
        [birthdate:App\Dto\ContactDto:private] => DateTimeImmutable Object
            (
                [date] => 1989-11-09 13:54:11.000000
                [timezone_type] => 3
                [timezone] => UTC
            )

        [salutation:App\Dto\ContactDto:private] => App\Normalizer\Value\StringValue Object
            (
                [value:App\Normalizer\Value\StringValue:private] => FEMALE
            )

        [marketingInformation:App\Dto\ContactDto:private] => App\Normalizer\Value\BooleanValue Object
            (
                [value:App\Normalizer\Value\BooleanValue:private] => 1
            )

    )
*/

Serializer default service configuration

The getters and setters for $salutation and $marketingInformation have StringValue and BooleanValue as the type.

This is because the ObjectNormalizer and ReflectionExtractor are enabled by default in symfony applications.

This ObjectNormalizer read and write in the object with the PropertyAccess component. This means that the ObjectNormalizer can access properties directly and through getters, setters, haters, issers, canners, adders and removers.

In the ReflectionExtractor (PropertyInfo component) it tries to get the type declaration from the mutator (set, add, remove), accessor (get, is, has, can) or constructor (in that order) based on the name of the property. If this is not possible, it will get the type from the property's type declaration.

So the ReflectionExtractor gets the type declaration from setSalutation(). If we were to define string as a type hint, $type for supportsDenormalization() would be of type string. Thus, we can no longer ensure that MappingTableNormalizer is applied to $salutation. This is because when the serialiser applies the normaliser to the properties, the property is not disclosed, just the value.

In my opinion, this default service configuration cannot be disabled or controlled by configuration parameters.

PropertyNormalizer and custom PropertyTypeExtractor

I am not fan of this Serializer default service configuration, because I would prefer that the getter and setter for $salutation and $marketingInformation does not have the ValueInterface as the type declaration.

Instead, I want the getters and setters to have the type declaration string or bool:

<?php

class ContactDto
{
    private ?StringValue $salutation = null;

    private ?BooleanValue $marketingInformation = null;

    public function getSalutation(): ?string
    {
        return $this->salutation?->getValue();
    }

    public function setSalutation(?string $salutation): void
    {
        $this->salutation = new StringValue($salutation);
    }

    public function isMarketingInformation(): ?bool
    {
        return $this->marketingInformation?->getValue();
    }

    public function setMarketingInformation(?bool $marketingInformation): void
    {
        $this->marketingInformation = new BooleanValue($marketingInformation);
    }
}

Therefore I need the PropertyNormalizer to read and write directly on the properties and a custom PropertyTypeExtractor to read only the type declaration from the properties.

Serializer service configuration for my custom requirements

services:
    'App\Service\CrmSerializerService':
        arguments:
            - '@crm_serializer'

    crm_serializer:
        class: 'Symfony\Component\Serializer\Serializer'
        arguments:
            $normalizers:
                - '@serializer.normalizer.datetime'
                - '@app.normalizer.mapping_table_normalizer'
                - '@crm_serializer.property_normalizer'
            $encoders: []

    crm_serializer.property_normalizer:
        class: 'Symfony\Component\Serializer\Normalizer\PropertyNormalizer'
        arguments:
            $nameConverter: '@serializer.name_converter.metadata_aware'
            $propertyTypeExtractor: '@crm_serializer.reflection_extractor'
            $defaultContext: '%serializer_default_context%'

    crm_serializer.reflection_extractor:
        class: 'Symfony\Component\PropertyInfo\PropertyInfoExtractor'
        arguments:
            $typeExtractors:
                - '@app.service.property_info.property_type_extractor'

    app.service.property_info.property_type_extractor:
        class: 'App\Service\PropertyInfo\PropertyTypeExtractor'

    app.normalizer.mapping_table_normalizer:
        class: 'App\Normalizer\MappingTableNormalizer'

I use the PropertyNormalizer instead of ObjectNormalizer, because it only reads and writes the value from the property.

Instead of the ReflectionExtractor I use a custom PropertyTypeExtractor (PropertyTypeExtractorInterface) which reads the type declaration only from the property.

The AdvancedNameConverterInterface is also needed to convert the name of the property (salutation <-> 46).

The $defaultContext is the serializer configuration with skip null values.

Then I populate the serializer with the required objects:

Normalizer
- DateTimeNormalizer
- MappingTableNormalizer
- PropertyNormalizer
Encoders
- Encoders are not needed at all, because only normalization and denormalization are done.

For the service configuration I use named arguments, because the other arguments are taken care of by the autowire and the default serializer service configuration can be used here.

I also have an example as an custum serializer object (CrmSerializer) that you can look at on github.

<?php

use App\Dto\ContactDto;
use Symfony\Component\Serializer\Serializer;
use Symfony\Component\Serializer\SerializerInterface;

class CrmSerializerService
{
    private Serializer $serializer; // Only Symfony Serializer

    public function __construct(SerializerInterface $serializer) {
        $this->serializer = $serializer;
    }

    public function normalize(ContactDto $contactDto): array
    {
        return $this->serializer->normalize($contactDto);
    }

    public function denormalize(array $data): ContactDto
    {
        return $this->serializer->denormalize($data, ContactDto::class);
    }
}

$contactDto = new ContactDto();
$contactDto->setSalutation('FEMALE');
/* other setter */
$contactDto->setMarketingInformation(true);

// Normalize ContactDto
$fields = $this->crmSerializerService->normalize($contactDto);
/*
    Array
    (
        [1] => Jane
        [2] => Doe
        [3] => jane.doe@example.com
        [4] => 1989-11-09
        [46] => 2
        [100674] => 1
    )
*/

Conclusion

The Symfony Serializer component is a powerful tool. I have many ideas on how to improve and simplify existing implementations with custom normalisers, especially in existing projects.

Full examples

Updates

Series name defined (May 5th 2023)
Update series name (May 8th 2023)
Add anchor for a deeplink (May 7th 2023)
Add bugfix note. (Nov 10th 2023)
Fix broken links (Dez 30th 2023)
Change GitHub Repository URL (Sep 4th 2024)

Use XInclude to organize Content elements in Page templates in SULU

André Laugks — Thu, 23 Feb 2023 21:33:56 +0000

With SULU, the content elements are defined with content types in the page templates. This can become confusing in a complex implementation with many page templates and content elements.

The keyword is reusability in order to avoid effort, errors and inconsistencies. For the name of the property for a text area, developers like to use bodytext, copytext or text.

A content element is, for example, an input field - like a headline - that stands alone or several input fields that are combined with a <block/>.

For this article SULU 2.5 is run in a Docker environment with PHP 8.2 (libxml 2.9.10).

The code examples are available on Github.

Content elements integrated in the page template

Here is an example where all content elements are defined directly in the page template.

To reduce the complexity for this article, I use a page template article.xml with the two content elements headline (content type: text_line) and bodytext (content type: text_editor).

<?xml version="1.0" ?>
<template 
   xmlns="http://schemas.sulu.io/template/template"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://schemas.sulu.io/template/template http://schemas.sulu.io/template/template-1.0.xsd">
   <!-- ... -->
   <properties>
      <!-- ... -->
      <property name="headline" type="text_line">
         <meta>
            <title lang="en">Headline</title>
         </meta>
      </property>
      <property name="bodytext" type="text_editor">
         <meta>
            <title lang="en">Bodytext</title>
         </meta>
      </property>
   </properties>
</template>

Content element in include files

To better organise the individual include files, I created the directory include within the templates directory. How you organise and name the include files is up to you.

Now the content elements are each transferred to the file headline.xml and bodytext.xml and stored in the directory include.

%kernel.project_dir%/config/templates/include/headline.xml

<?xml version="1.0" ?>
<properties xmlns="http://schemas.sulu.io/template/template"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://schemas.sulu.io/template/template http://schemas.sulu.io/template/template-1.0.xsd">
    <property name="headline" type="text_line">
        <meta>
            <title lang="en">Headline</title>
        </meta>
    </property>
</properties>

%kernel.project_dir%/config/templates/include/bodytext.xml

<?xml version="1.0" ?>
<properties xmlns="http://schemas.sulu.io/template/template"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://schemas.sulu.io/template/template http://schemas.sulu.io/template/template-1.0.xsd">
    <property name="bodytext" type="text_editor">
        <meta>
            <title lang="en">Bodytext</title>
        </meta>
    </property>
</properties>

In order to be able to use the content elements in article.xml, XInclude is used. XInclude is a recommendation of the W3C to refer to XML or text documents within XML documents in order to replace the content with the referenced file.

%kernel.project_dir%/config/templates/pages/article.xml

<?xml version="1.0" ?>
<template xmlns="http://schemas.sulu.io/template/template"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns:xi="http://www.w3.org/2001/XInclude"
          xsi:schemaLocation="http://schemas.sulu.io/template/template http://schemas.sulu.io/template/template-1.0.xsd">
    <!-- ... -->
    <properties>
        <!-- ... -->
        <xi:include href="../include/headline.xml"
                    xpointer="xmlns(sulu=http://schemas.sulu.io/template/template) xpointer(/sulu:properties/sulu:property)">
            <xi:fallback>An error occurred</xi:fallback>
        </xi:include>
        <xi:include href="../include/bodytext.xml"
                    xpointer="xmlns(sulu=http://schemas.sulu.io/template/template) xpointer(/sulu:properties/sulu:property)">
            <xi:fallback>An error occurred</xi:fallback>
        </xi:include>
    </properties>
</template>

I do not want to go into great detail in this article about how XML-Namespaces, XInclude, XPointer and XPath work, however, the principle is quite simple.

The default namespace in the include files is specified with the attribute xmlns, because only tags from SULU are used in the XML documents. The page template has the same default namespace, however, these have nothing to do with each other.

<xi:include href="%REFERENCE%" xpointer="xmlns(%PREFIX%=%URI%) xpointer(%XPATH%)" />

%REFERENCE% is the relative reference to the file to be included. Then we define the %PREFIX% and %URI% of the namespace. With XPointer it is not the prefix but the URI that is important. In the example I use sulu, but it could also be banana. For the URI we use the default namespace from the include files. For %XPATH% we use another XPath expression to find the element.

All content elements in an include file

XPath gives us the possibility to find elements in the documents. We could therefore store all content elements in a document and find them with an XPath expression using the name of elements or properties.

%kernel.project_dir%/config/templates/pages/elements.xml

<?xml version="1.0"?>
<properties xmlns="http://schemas.sulu.io/template/template"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://schemas.sulu.io/template/template http://schemas.sulu.io/template/template-1.0.xsd">
    <property name="headline" type="text_line">
        <meta>
            <title lang="en">Headline</title>
        </meta>
    </property>
    <property name="bodytext" type="text_editor">
        <meta>
            <title lang="en">Bodytext</title>
        </meta>
    </property>
</properties>

%kernel.project_dir%/config/templates/pages/article.xml

<?xml version="1.0" ?>
<template xmlns="http://schemas.sulu.io/template/template"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns:xi="http://www.w3.org/2001/XInclude"
          xsi:schemaLocation="http://schemas.sulu.io/template/template http://schemas.sulu.io/template/template-1.0.xsd">
    <!-- ... -->
    <properties>
        <!-- ... -->

        <!-- Example: Complete path with attribute -->
        <xi:include href="../include/elements.xml"
                    xpointer="xmlns(sulu=http://schemas.sulu.io/template/template) xpointer(/sulu:properties/sulu:property[@name = 'headline'])">
            <xi:fallback>An error occurred</xi:fallback>
        </xi:include>

        <!-- Example: Find element only by attribute -->
        <xi:include href="../include/elements.xml"
                    xpointer="xmlns(sulu=http://schemas.sulu.io/template/template) xpointer(//*[@name = 'bodytext'])">
            <xi:fallback>An error occurred</xi:fallback>
        </xi:include>
    </properties>
</template>

Observation

In another tutorial, in preparation for this article, I saw an example with XInclude which does not work in my development environment. In the tutorial, no namespace and XML schema were defined in the file to be included and no xpointer was defined in the article.xml in <xi:include/>.

Content element

<?xml version="1.0" ?>
<property name="headline" type="text_line">
    <meta>
        <title lang="en">Headline</title>
    </meta>
</property>

Page template

<?xml version="1.0" ?>
<template xmlns="http://schemas.sulu.io/template/template"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns:xi="http://www.w3.org/2001/XInclude"
          xsi:schemaLocation="http://schemas.sulu.io/template/template http://schemas.sulu.io/template/template-1.0.xsd">
    <!-- ... -->
    <properties>
        <!-- ... -->
        <xi:include href="../include/headline.xml"/>
    </properties>
</template>

The following exception is provided:

Uncaught PHP Exception Symfony\Component\Config\Util\Exception\XmlParsingException: "[ERROR 1871] Element 'property': This element is not expected. (in /home/wwwroot/config/templates/pages/article.xml - line 2, column 0)" at /home/wwwroot/vendor/symfony/config/Util/XmlUtils.php line 95 {"exception":"[object] (Symfony\\Component\\Config\\Util\\Exception\\XmlParsingException(code: 0): [ERROR 1871] Element 'property': This element is not expected. (in /home/wwwroot/config/templates/pages/article.xml - line 2, column 0) at /home/wwwroot/vendor/symfony/config/Util/XmlUtils.php:95)"} []

The exception "[ERROR 1871] Element 'property': This element is not expected. (in /home/wwwroot/config/templates/pages/article.xml - line 2, column 0)" is not from Symfony or SULU, but from libxml which implemented PHP.

I have not looked further into this error message or debugged it because on the one hand the XInclude declaration seems unclean and on the other hand the autocompletion of the IDE cannot be used due to the missing XML schema definition in the include files.

My suspicion is that the developer was using an older version of libxml and this might have caused it to work. This is because not all XML processors and versions of these XML processors always fully support the standard defined by the W3C.

Fallback

I have used <xi:fallback /> in my examples, but it is an optional element. I use it because it also helps with debugging, it is cleaner and it tells users that the include is not working.

Because XML processors do not have to provide an error message if the file cannot be included.

DEV Community: André Laugks

Using nested annotations for key-value pairs in a custom annotation

Introduction

Annotations

Custom Annotation @MapItem

Custom Annotation @MapItems

Functional Test

Conclusion

Pro

Contra

Full example

Related article

Using a HashMap in a custom Annotation

Introduction

Enum Constant

Custom Annotation @MapItems

Functional Test

Conclusion

Pro

Contra

Full example

Problems with webfonts (an article from 1998)

Problems with downloadable font files

1. Possibility: Load Active X Control in Internet Explorer

2.1 Option: Load browser query and font file dynamically

2.2 Load separate CSS file

"Risks and side effects" of downloadable fonts

Mitigating risks and side effects of downloadable font files

Further links to downloadable font files

Create a custom Jackson JsonSerializer und JsonDeserializer for mapping values

The required packages

Minimal structure of a custom JsonSerializer and JsonDeserializer

Custom Type

ContactDto

Custom Annotation @MappingTable and Enum Constants for the MappingTable

Annotation @MappingTable

Enum Constants

Field definitions

Reading and writing on the fields

FieldIDs

Define custom JsonSerializer and JsonDeserializer

Skip null values

Ignore unknown properties

MappingValueSerializer and MappingValueDeserializer

MappingValueSerializer

MappingValueDeserializer

MappingTableMapReader

Functional Test

Full example on GitHub

Related article

Updates

Create a custom JMS Serializer handler for mapping values

Create the handler

Define the FieldID and MappingTable

Register the handler

Symfony JMSSerializerBundle

JMS Serializer standalone

Normalize and denormalize

Links

Updates

Create a custom Symfony Normalizer and Denormalizer for mapping values

The requirements and history

Built-in mapping in the API-Client

Use a custom Attribute

Using the Symfony serializer component and creating a custom normalizer

FieldIDs with attribute #[SerializedName]

Normalize/Denormalize and Serialize/Deserialize

⚠️ Note: #[SerializedName] with a number as name in Symfony 6.2

Create a custom Normalizer

Attribute #[Context] for the MappingTable

Register the custom normalizer

Skip null values

Normalize the ContactDto to an array

Denormalize the array to a ContactDto

Serializer default service configuration

PropertyNormalizer and custom PropertyTypeExtractor

Serializer service configuration for my custom requirements

Conclusion

Full examples

Updates