Ivan Goremykin

Posted on Feb 2, 2023 • Edited on Apr 23, 2023

Adding Codable Conformance to Union with Metaprogramming

#mobile #ios #swift #computerscience

TL;DR

We add Codable conformance to a set of enums — Union2, Union3, etc. by representing each UnionX in KeyedEncodingContainer as a “value” and a “type”. The choice of coding scheme is team-specific and configurable via Sourcery template arguments. The code is generated using a Sourcery template written in Swift. All the source code, including a Swift playground, Sourcery templates, configuration files, and scripts, is available on GitHub. You can also run the code on Repl.it.

Preface

In the article Adding Union to Swift with Metaprogramming we have generated a set of enums — Union2, Union3, etc. that act like a disjoint set. For every UnionX, we also provide a selection of helper methods: conformance to the standard Swift protocols, higher-order functions, etc. Adding Encodable and Decodable conformance to UnionX is a significant topic on its own, which is why it’s covered in a separate article.

Motivation

The essence of many mobile applications is manipulating lists of elements. Sometimes, we need to encode and decode lists where elements are not of the same type.

Encoding a type that conforms to Encodable is trivial. However, decoding it is not — since we don’t know which type to decode beforehand. In the following example, we have an array with 2 objects: the first one is of type Album, and the second one is of type Artist:

[
    {
        "id": "808a0004-df02-4cf6-b5fb-caec1c155420",
        "name": "Tilt",
        "release-date": "1995-05-08",
        "artistId": "dff5a5ad-6185-47ab-ae42-c72045bfa38a"
    },
    {
        "id": "2c51f969-f145-4a65-b9bc-3ed4c0840b0d",
        "name": "Mark Hollis",
        "biography": "Mark David Hollis was an English musician, the main vocalist and songwriter of the band Talk Talk."
    }
]

If we want to decode this array, how do we know which object should be decoded as Album and which — as Artist?

What other developers do

Brute force

To decode an array of non-homogeneous values, some implementations use a brute-force approach:

they try to decode the object as TypeA
if they fail, they try to decode it as TypeB
and so on until they either succeed or run out of types to decode

Not only this approach has obvious performance implications, but there is no guarantee that a set of type’s CodingKeys can act as a composite key, i.e. uniquely identify the type. For instance, if TypeA has a subset of TypeB’s keys, we can mistakenly decode JSON object that represents TypeB as TypeA:

struct Type A {
   let id: UUID
   let name: String
}

struct Type B {
   let id: UUID
   let name: String
   let expiredOn: Date
}

This JSON object can be decoded both as TypeA and as TypeB.

{
   “id”: “7a910713-dcbd-4f6e-a739-2adb3d445cdb”,
   “name”: “Trial Subscription”,
   “expiredOn”: “2012-12-21”,
}

Protocol Buffers

ProtocolBuffers’ OneOf message addresses the case of having a message with many fields where at most one field will be set at the same time.

message SampleMessage {
  oneof sample_oneof {
    string name = 4;
    SubMessage sub_message = 9;
  }
}

It supports fields of any type, except map fields and repeated fields. This concept lacks the disadvantages of brute force and will underlie our approach.

What we’re going to do

We’re going to represent UnionX using 2 coding keys:

“value” that contains the encoded object carried by UnionX
“type” that uniquely identifies the wrapped type

We will implement a Sourcery template for adding Codable conformance to UnionX, where X is the size of the enum. X is going to be an argument of the Sourcery template.

We will go through various aspects of it using Union2 as an example.

Coding Scheme

A modified version of the previous example with an array of non-homogeneous values will look like this:

[
    {
        "type": "album",
        "value": {
            "id": "808a0004-df02-4cf6-b5fb-caec1c155420",
            "name": "Tilt",
            "release-date": "1995-05-08",
            "artistId": "dff5a5ad-6185-47ab-ae42-c72045bfa38a"
        },
    {
        "type": "artist",
        "value": {
            "id": "2c51f969-f145-4a65-b9bc-3ed4c0840b0d",
            "name": "Mark Hollis",
            "biography": "Mark David Hollis was an English musician, the main vocalist and songwriter of the band Talk Talk."
        }
    }
]

Each UnionX object will be represented in JSON as follows:

{
    "<Type ID Key>": "<Type ID Value>",
    "<Value Key>": Value
}

The choice of Type ID Key, Type ID Value, and Value Key depends on each particular development team. The following factors should be taken into consideration:
1. The choice of names of Type ID Key and Value Key attributes

in this example it’s “type” and “value”
it could be “type_id” and “wrapped” or something else

2. The choice of Type ID Value

it can be human-readable — String (“album”, “artist”, etc.)
or not human-readable — UUID, Int, etc.
each has its pros and cons

3. The UnionX coding scheme should be the same across the whole project

if your team has a couple of endpoints that return a non-homogeneous array of JSON objects, they all should follow the same scheme
the JSON scheme is highly likely to be shared by multiple platforms (iOS, Android, Web), so there should be an agreement between all the stakeholders

Because the choice of Type ID Key, Type ID Value, and Value Key is team-specific, our Sourcery template is going to accept following arguments:

Type ID Key — e.g. “type”, “id”, “type_id”
Type ID Value — e.g. “String”, “UUID”, “Int”
Value Key — e.g. “value”, “wrapped”

Codable

Now that we have agreed on the coding scheme, let’s take a look at Swift code for encoding and decoding unions.

First, let’s define the type of Type ID (e.g. “String”, “UUID”, “Int”) — UnionTypeID:

typealias UnionTypeID = String // 'String' comes from the Sourcery template arguments

This is how we generate this line of code in our meta-code.

Next, we want to get Type ID Value (e.g. “album”, “artist”) from all types wrapped by UnionX. E.g. if there is Union2<Album, Artist>, we want both Album and Artist to provide Type ID Value. This requirement will come in the form of protocol:

protocol UnionIdentifiable {
    static var unionTypeID: UnionTypeID { get }
}

Both Album and Artist are going to conform to UnionIdentifiable:

extension Artist: UnionIdentifiable {
    static let unionTypeID: UnionTypeID = "artist"
}

extension Album: UnionIdentifiable {
    static let unionTypeID: UnionTypeID = "album"
}

We’re not using Swift’s Identifiable because it identifies instances, not types, and also because it has associatedtype requirements, i.e. it will not work if Artist returns ID of type String and Album returns ID of type UUID.

We will also need a CodingKey for adding Encodable and Decodable conformance to UnionX. For every UnionX object, we will need to encode 2 attributes — Type ID Key and Value Key; hence we define 2 coding keys:

struct UnionCodingKey: CodingKey {
    var stringValue: String
    init?(stringValue: String) { self.stringValue = stringValue }

    var intValue: Int? { return nil }
    init?(intValue: Int) { return nil }

    static let unionTypeID = UnionCodingKey(stringValue: "type")! // "type" comes from the Sourcery template arguments
    static let wrappedValue = UnionCodingKey(stringValue: "value")! // "value" comes from the Sourcery template arguments
}

The respective metaprogramming code can be found here. Note that UnionTypeID, UnionIdentifiable, and UnionCodingKey will be shared across Union2, Union3, etc.

Encoding

Now that we have a coding key, let’s encode our Union:

extension Union2: Encodable where Item0: Encodable & UnionIdentifiable, Item1: Encodable & UnionIdentifiable {
    func encode(to encoder: Encoder) throws {
        var unionContainer = encoder.container(keyedBy: UnionCodingKey.self)

        try unionContainer.encode(unionTypeID, forKey: .unionTypeID)
        try unionContainer.encode(innerEncodable, forKey: .wrappedValue)
    }

    private var unionTypeID: UnionTypeID {
        switch self {
        case .item0:
            return Item0.unionTypeID

        case .item1:
            return Item1.unionTypeID
        }
    }

    private var innerEncodable: Encodable {
        switch self {
        case .item0(let item0):
            return item0

        case .item1(let item1):
            return item1
        }
    }
}

The corresponding metaprogramming code can be found here.

A few notes about the generated code above:

if we want to make our UnionX conform to Encodable, we will need all of its wrapped types to conform not only to Encodable, but also to our UnionIdentifiable
to follow the single-responsibility principle and make our encode(to:) easy to read, we generated 2 computed properties — unionTypeID and innerEncodable
both of them should be not accessed outside of the extension and hence are marked private

Decoding

Decoding UnionX will look like this:

extension Union2: Decodable where Item0: Decodable & UnionIdentifiable, Item1: Decodable & UnionIdentifiable {
    func init(from decoder: Decoder) throws {
        let unionContainer = try decoder.container(keyedBy: UnionCodingKey.self)

        let unionTypeID = try unionContainer.decode(UnionTypeID.self, forKey: .unionTypeID)

        switch unionTypeID {
        case Item0.unionTypeID:
            self = .item0(try unionContainer.decode(Item0.self, forKey: .wrappedValue))

        case Item1.unionTypeID:
            self = .item1(try unionContainer.decode(Item1.self, forKey: .wrappedValue))

        default:
            throw UnionDecodingError.unknownUnionTypeID(unionTypeID)
        }
    }
}

The respective metaprogramming code can be found here.

A few notes about the generated code above:

if we want to make our UnionX conform to Decodable, we will need all of its wrapped types to conform not only to Decodable, but also to our UnionIdentifiable
in case we fail to match unionTypeID with any known type ID, we will throw UnionDecodingError:

enum UnionDecodingError: Error {
    case unknownUnionTypeID(UnionTypeID)
}

UnionDecodingError is shared across Union2, Union3, etc.

End-to-end example

Now that we have all the code, we can take a look at how it’s going to be used:

// Encoding
let sample: [U2<Album, Artist>] = [
    .init(Album.sample0),
    .init(Album.sample1),
    .init(Album.sample2),
    .init(Artist.sample0),
    .init(Artist.sample1)
]

let encoder = JSONEncoder()

let encodedSample = try encoder.encode(sample)

// Decoding
let decoder = JSONDecoder()

let decodedSample = try decoder.decode(
    [U2<Album, Artist>].self,
    from: encodedSample
)

print(sample == decodedSample) // true

Discussion

We have written meta code for adding Codable conformance to a set of UnionX types.

You can check both meta-code and generated code

on GitHub, where it is available as a Swift playground and a set of scripts and configuration files for running Sourcery
or on Repl.it, where you can run the whole thing in a browser

An interesting implication of this approach is that it is possible to encode UnionX and then decode it as UnionX+1:

let arrayOfU2: [U2<Album, Playlist>] = try decoder.decode(
    [U2<Album, Playlist>].self,
    from: encodedSample
)

let arrayOfU3 = try decoder.decode(
    [U3<Album, Playlist, Artist>].self,
    from: encodedSample
)

arrayOfU2.compactMap0() == arrayOfU3.compactMap0() // Compare two arrays [Album]
arrayOfU2.compactMap1() == arrayOfU3.compactMap1() // Compare two arrays [Artist]

… which makes our solution open for extension.

Acknowledgments

I would like to thank Vyacheslav Shakaev for his constructive criticism and valuable comments on the draft version of this text.

Top comments (7)

Pilar Fernandez Mutti • Jun 5 '23

Super useful article in addition to UnionX implementation.
It is very well explained, including the example!
Thank you

Ivan Goremykin • Jun 5 '23

Hello @pilar_fernandezmutti_5b7, thank you for reading!

Stay tuned for the next article in the Union series where I will be exploring Apple's Combine framework 🤓

Dmitry • Jun 6 '23 • Edited

Very neat use case for metaprogramming. Now I wonder, can the generation of code for different unions of the same size be avoided? I.e. if I have Union<A,B> and Union<X,Y> - can I reuse Union<A,B>? I feel like it would be either super tricky or impossible tho)
Thank you for useful info!

Ivan Goremykin • Jun 14 '23

@dmitry, thank you 🙏

As for using Union for types A and B, and then for X and Y — we don't need to generate a version of Union for each wrapped type combination.

We will be using Union2 in both cases. You can check the source code of Union2 here.

Dmitrii Morozov • Jun 5 '23

I always felt bad trying to use bruteforce approach in decode, now I'll use this trick, thanks very helpful

Ivan Goremykin • Jun 5 '23

You're welcome @mtmorozov 🙂

Comment deleted

DEV Community