How Not to Attempt an April Fool’s Day Prank

Jason R Tibbetts — Mon, 01 Apr 2019 19:25:48 +0000

In light of Ben’s post about an April Fool’s Day prank gone wrong, it reminded me of the biggest failed developer prank I’ve ever experienced. I once worked on a mission-critical application for a government agency. It was classified at a very high level. I was one of the developers who was cleared to view classified data, but others on the team weren’t, so they worked with dummy data.

One morning, I got a frantic call from the project manager begging me to get to the office as soon as possible because some time overnight, the application looked broken. Not only was the data corrupted, but so were all of the UI elements. It was completely unusable. Even worse, some of the data in the database (several gigabytes’ worth) was also corrupted.

I got to the office and took a look. My project lead was shaking with anxiety. There was a pattern to the corrupted data, and I it took me about 10 seconds to figure out what was wrong. “What’s today’s date?” I asked the lead.

“April 1st. Why?”

I told him to take another look.

“What? I just see gibberish.”

“It’s all backwards,” I said. Sure enough, every string was reversed. The leads anguish turned to anger.

The next step was to figure out how, and then, who did it. I figured out the latter pretty quickly, and it was the last person I’d expected to do anything remotely humorous--a 25-year veteran of the group. He didn't have a clearance, so he never saw the corrupted data.

The how was the most interesting part. The developer had somehow hooked into the Java Swing UI to reverse the strings, but then those changes to the editable fields were auto-saved. He had to issue a hotfix for both the app and the database, without being able to access them to apply the changes himself, and I’m sure that he was more pissed about having to walk me through the fixes than he was contrite about having messed things up so badly. In the end, all the data was restored, and the episode lasted, at most, a couple of hours.

So, Dear Reader, please think twice, or maybe three times, before pulling off a stunt like this at work, especially on production, mission-critical apps!

Decoding Formatted JSON Dates in Swift

Jason R Tibbetts — Fri, 08 Feb 2019 15:34:34 +0000

If you've done any Web, mobile, or desktop programming in the past decade (and would you even be here if you hadn't?), you've used JSON--the lingua franca for serializing data across devices and platforms. JSON's popularity lies in its simplicity: it supports only a handful of data types, like strings, numerics, Booleans, arrays, and dictionaries, and is generally easy for humans to read. But for strongly-typed languages like Swift, this simplicity could lead to suboptimal parsing strategies. Xcode 9's Swift SDK introduced an elegant way to handle parsing, and that was with the Codable protocol. When you declare a class, struct, or enum as Codable, the Swift compiler implicitly generates three things: init(from: Decoder), encode(to: Encoder), and a CodingKey enumeration, which maps JSON key values to the names of your Codable's properties. To make your data type Codable, you simply declare that it implements Codable, and ensure that all of its properties are themselves Codable or are one of the supported types:

Boolean
String
Int
Float
Date
Data
URL
arrays of Codable types
dictionaries of Codable types
optionals of all of the above

By default, the JSONDecoder can parse Dates only in their raw form--a TimeInterval (=Double) of their milliseconds since the reference date: 12:00 a.m. UTC² on January 1st, 2000. This article's focus is on supporting custom Date formats, and the journey I took from naïve implementation to full understanding.

Sunrise, Sunset

"Sunrise, Sunset" from The Fiddler on the Roof (1971)

I've recently been working with some REST APIs for looking up sunrise and sunset times for a given latitude and longitude. One of them, Sunrise-Sunset.org, returns the following JSON when called with https://api.sunrise-sunset.org/json?lat=41.948437&lng=-87.655334&date=2019-02-14³ (omitting some lines for clarity):

{
    "results":
    {
        "sunrise":"12:59:22 PM",
        "sunset":"11:09:45 PM"
    }
}

I created the following Codable structures, which I'll call

Approach #1: The Naïve Approach, aka How It Should Work

public struct SunriseSunsetResponse: Codable {
    public var results: SunriseSunset
}

public struct SunriseSunset: Codable {
    public var sunrise: Date
    public var sunset: Date
}

SunriseSunset is a nested element that represents the value of the "results" element, and both it and the SunriseSunsetResponse have to implement Codable. Now I'll create an instance of SunriseSunsetResponse by calling the API and parsing it:

if let url = URL(string: "https://api.sunrise-sunset.org/json?lat=41.948437&lng=-87.655334&date=2019-02-04") {
    do {
        let responseData = try Data(contentsOf: url)
        let response = try JSONDecoder().decode(SunriseSunsetResponse.self, from: responseData)
        let sunriseSunset = response.results
        print("Daylight duration (in seconds): \(sunriseSunset.sunset.timeIntervalSince(sunriseSunset.sunrise))")
    } catch {
        print(error)
    }
}

And...it fails. The error tells says

typeMismatch(Swift.Double, Swift.DecodingError.Context(
    codingPath: [CodingKeys(stringValue: "results", intValue: nil),
                 CodingKeys(stringValue: "sunrise", intValue: nil)],
    debugDescription: "Expected to decode Double but found a string/data instead.",
    underlyingError: nil))

Did you spot the problem? sunrise and sunset properties are Dates, but the parser found the strings "12:59:22 PM" and "11:09:45 PM" instead. How can I fix this, without manipulating the JSON data in some way? Here's what I did:

Approach #2: Storing the dates as `String`s

Let's refactor SunriseSunset to expect date strings, like

public struct SunriseSunset: Codable {
    public var sunrise: String
    public var sunset: String
}

This shifts the responsibility for parsing the strings into Dates to the caller. The downside is that if the SunriseSunset object is used in multiple places, you may wind up with many identical parsing calls. Even if you create a single DateFormatter instance and used it in multiple places, you'd still wind up violating the dreaded DRY (Don't Repeat Yourself) principle. There must be a better way. Let's try

Approach #3: Keep the `String` properties, and add corresponding computed `Date` properties

I want to simplify how my SunriseSunset gets used, so why not make it responsible for parsing the dates itself? I'll add a DateFormatter property, plus computed Date properties. This sounds better than the naïve, String-based approach, even though the Date versions should be Optional, because that's what DateFormatter.date(from:) returns:

public struct SunriseSunset: Codable {
    private var dateFormatter: DateFormatter

    public var sunrise: String

    public var sunriseDate: Date? {
        return dateFormatter.date(from: sunrise)
    }

    public var sunset: String

    public var sunsetDate: Date? {
        return dateFormatter.date(from: sunset)
    }
}

This is starting to look kind of ugly, but it should suit my purposes. But now it won't compile! Remember that every property of a Codable type that you want to convert to & from JSON must be a type that itself implements Codable, and DateFormatter does not. There is a workaround for this, which we'll call

Approach #4: Using custom `CodingKeys`

This way is to define a enumeration of CodingKeys for your Codable type. The CodingKeys enumeration must having a raw type of String and CodingKey (note the singular), in that order. The Swift compiler generates this for you from your properties' names for free, but if your property names don't exactly match the JSON data, or, as in this case, you don't want all of your properties to be parsed from JSON data, you must add your own. So now we'll try:

public struct SunriseSunset: Codable {
    private var dateFormatter: DateFormatter

    public var sunrise: String

    public var sunriseDate: Date? {
        return dateFormatter.date(from: sunrise)
    }

    public var sunset: String

    public var sunsetDate: Date? {
        return dateFormatter.date(from: sunset)
    }

    private enum CodingKeys: String, CodingKey {
        case sunrise
        case sunriseDate
        case sunset
        case sunsetDate
    }
}

Note that I've omitted dateFormatter from the custom keys. But again, this won't compile. The compiler barfs numerous errors, the most important of which are:

error: type 'SunriseSunset4' does not conform to protocol 'Decodable'

public struct SunriseSunset4: Codable {

note: protocol requires initializer 'init(from:)' with type 'Decodable'
public init(from decoder: Decoder) throws

error: type 'SunriseSunset4' does not conform to protocol 'Encodable'

public struct SunriseSunset4: Codable {

note: protocol requires function 'encode(to:)' with type 'Encodable'
public func encode(to encoder: Encoder) throws

What these are telling you (not very clearly, IMHO) is that if you have properties that should not be handled by the JSONDecoder/JSONEncoder, then you have to supply a custom initializer and encoding function. Apple's documentation really doesn't help much. It says,

> Omit properties from the `CodingKeys` enumeration if they won't be present
> when decoding instances, or if certain properties shouldn't be included in an
> encoded representation. A property omitted from `CodingKeys` needs a default
> value in order for its containing type to receive automatic conformance to
> `Decodable` or `Codable`.

This sounds like you should be able to assign dateFormatter a DateFormatter instance when it's declared, like

> let dateFormatter = DateFormatter()

but you can't. The only way is to implement the initializer and encoding function. If you're thinking to yourself that this really defeats the purpose of using Codable in the first place, which is to let the Swift compiler generate the CodingKeys, initializer, and encoding functions, then you're completely correct. "There must be a better way!" I said. And there is! It's

Approach #5: `dateDecodingStrategy` with a custom `DateFormatter`

JSONDecoder has a property called dateDecodingStrategy, of type JSONDecoder.DateDecodingStrategy, which allows you to change how dates are parsed. This an enum with 6 cases:

deferredToDate (default): This treats Dates as Double values that indicate the date's number of milliseconds since the reference date (see above)
iso8601: The best way to format dates, e.g. "2019-02-04T12:59:22+00:00"
formatted(DateFormatter): Allows you to use custom DateFormatter instance
custom(@escaping (Decoder) -> Date): Allows you to specify a custom block for parsing
millisecondsSince1970: Like the default deferredToDate option, but calculates dates from the beginning of the Unix epoch (i.e. January 1st, 1970)
secondsSince1970: Like millisecondsSince1970, but in seconds, not milliseconds

Thankfully, "12:59:22 PM" happens to be an exact match for DateFormatter.TimeStyle.medium, so I'll configure my decoder accordingly:

let decoder = JSONDecoder()
let dateFormatter = DateFormatter()
dateFormatter.timeStyle = .medium
decoder.dateDecodingStrategy = .formatted(dateFormatter)

Now it prints the answer I expected:

Daylight duration (in seconds): 36623.0

I'm done, right? Not quite. Examining the returned date further, I find that although the time is what I expected, the date is January 1st, 2000! That doesn't do me much good if I want to calculate how many seconds have elapsed since sunrise today! Now I have to normalize the returned times to today's date, and that's a little trickier. One way is to get the interval from the reference date to midnight today, and add that to the parsed time.

let midnightThen = Calendar.current.startOfDay(for: sunriseSunset.sunrise)
let millisecondsToSunrise = sunriseSunset.sunrise.timeIntervalSince(midnightThen)

let midnightToday = Calendar.current.startOfDay(for: Date())
let normalizedSunrise = midnightToday.addingTimeInterval(millisecondsToSunrise)

However, there's no way to do this kind of transformation simply by using a custom DateFormatter instance, so we're back to the original problem of duplicating time-normalization calls throughout my app. Well, it turns out that there is a dateDecodingStrategy that can do this, and that's

Approach #6: Using a custom `dateDecodingStrategy` block

One of the JSONDecoder.DateDecodingStrategy enum cases is custom, which takes an associated block that gets a JSONDecoder instance and returns a Date. So let's put the previous date-manipulation code into that block, like

dateDecodingStrategy = .custom({ (decoder) -> Date in
    // Parse the date using a custom `DateFormatter`
    let container = try decoder.singleValueContainer()
    let dateString = try container.decode(String.self)
    let date = self.dateFormatter.date(from: dateString)

    let midnightThen = Calendar.current.startOfDay(for: date)
    let millisecondsFromMidnight = date.timeIntervalSince(midnightThen)

    let midnightToday = Calendar.current.startOfDay(for: Date())
    let normalizedDate = midnightToday.addingTimeInterval(millisecondsFromMidnight)

    return normalizedDate
})

Note the first three statements in the block. The first two show how to get a single String value from decoder. But what is this decoder instance? It's of type Decoder (not JSONDecoder!), and it holds a single element--in this case, a JSON value string. (If your JSON contains an array or dictionary of Dates that need to be manipulated, you would change the container and decoded types accordingly.)

Is that it? Are we done? Not quite. Note that this custom decoding strategy still needs a DateFormatter instance. DateFormatter instances are expensive to create, so I'll create one and assign it to a property of the class that sets up this dateDecodingStrategy. To keep things relatively self-contained, I subclassed JSONDecoder, like so:

class NormalizingDecoder: JSONDecoder {

    /// The formatter for date strings returned by `sunrise-sunset.org`.
    /// These are in the `.medium` time style, like `"7:27:02 AM"` and
    /// `"12:16:28 PM"`.
    let dateFormatter: DateFormatter
    let calendar = Calendar.current

    override init() {
        super.init()
        dateFormatter = DateFormatter()
        dateFormatter.timeStyle = .medium
        keyDecodingStrategy = .convertFromSnakeCase
        dateDecodingStrategy = .custom { (decoder) -> Date in
            let container = try decoder.singleValueContainer()
            let dateString = try container.decode(String.self)
            let date = self.dateFormatter.date(from: dateString)

            if let date = date {
                let midnightThen = calendar.startOfDay(for: date)
                let millisecondsFromMidnight = date.timeIntervalSince(midnightThen)

                let today = Date()
                let midnightToday = calendar.startOfDay(for: today)
                let normalizedDate = midnightToday.addingTimeInterval(millisecondsFromMidnight)

                return normalizedDate
            } else {
                throw DecodingError.dataCorruptedError(in: container,
                                                       debugDescription:
                    "Date values must be formatted like \"7:27:02 AM\" " +
                    "or \"12:16:28 PM\".")
            }
        }
    }
}

Using this custom JSONDecoder, our Codable can once again look like what we wanted in Approach #1, namely

public struct SunriseSunsetResponse: Codable {
    public var results: SunriseSunset
}

public struct SunriseSunset: Codable {
    public var sunrise: Date
    public var sunset: Date
}

Dilbert achieves Nerdvana
With this approach, you can do even more:

Adjust times for time zone offsets
Handle dates that may be in one of several acceptable formats
Handle arrays and dictionaries of formatted Dates

If you've made it this far, thank you for reading! This is my first public technical writeup, despite being a professional developer since 1996.

Footnotes

¹ Technically speaking, Codable is a typealias of Encodable and Decodable.
² Coordinated Universal Time, better known as Greenwich Mean Time (GMT).
³ If you're a Blues Brothers fan, you may recognize these as the coordinates for Elwood Blues's address.

DEV Community: Jason R Tibbetts

How Not to Attempt an April Fool’s Day Prank

Decoding Formatted JSON Dates in Swift

Sunrise, Sunset

Approach #1: The Naïve Approach, aka How It Should Work

Approach #2: Storing the dates as Strings

Approach #3: Keep the String properties, and add corresponding computed Date properties

Approach #4: Using custom CodingKeys