How to Unit Test Swift CLI applications in XCode

If you are short of time, you can jump directly to the solution to the problem skipping the description of failed attempts to solve it.

I have recently started solving some programming problems in Swift and wanted to have the convenience of Unit Testing my code to decrease the need for manual testing and make sure my code stays correct while I keep modifying it.

Initial Attempt

CLI Project Setup

Creating an iOS or Mac OS X app just to solve simple problems seemed overkill for me, so I opted for the Command Line Interface (CLI) template in Xcode 13.

The first thing I noticed was the absence of a Test Target which you usually get for iOS/Mac applications out of the box.

I have crafted some code to test whether a string is a palindrome in a main.swift file and made sure it compiles and runs without obvious runtimes crashes.

func isPalindrome<S: StringProtocol>(_ str: S) -> Bool {
    guard nil != str.first else { return false }

    var headIdx = str.startIndex
    var tailIdx = str.index(before: str.endIndex)

    while headIdx <= tailIdx {
        if str[headIdx] != str[tailIdx] { return false }
        if headIdx == tailIdx { break }

        headIdx = str.index(after: headIdx);
        tailIdx = str.index(before: tailIdx);
    }
    return true;
}

Adding Test Target

Now I wanted to add a Test Target so I can write unit tests to extensively test my function.

The first suspicious thing is that I cannot add my CLI application as a Target to be tested

Upon trying to run the test I got an error Cannot find 'isPalindrome' in scope which seems correct on the surface - I didn't import my CLI application into the Test Target.

I did import my CLI App Target and tried again

@testable import cli_unit_tests

No luck this time either! I got a linker error which is a bad sign.

Solution

To save you some time of interesting discoveries - there is no direct way to Unit Test a CLI application in XCode. At least I failed to find one.

On the surface you have just two options:

1. write your CLI application code without unit tests, which may be a terrible idea in many circumstances.
2. Avoid writing a CLI application in Swift in the first place, which may be against what you wanted to achieve.

Fortunately, there is a third option - move all the code you need to Test in a Framework. By doing this you achieve:

• The CLI Application can use all the code from the Framework
• You can Unit Test all the code in the Framework
• You get a good boundary separation inside your code

Below is the sequence of steps to properly add Framework (not Static Library) to a CLI application to allow Unit Testing of your code.

You can get the source codes of the project we will be creating below on Github.

Optional Preparation

If you tried to follow the steps I did in the initial attempt make sure to Delete the unit_tests Target that was created in the previous steps. Or better start a new project from scratch altogether.

You will also need to delete the Scheme that was created by XCode for the unit_tests Target.

For the sake of the clean experiment, I also deleted the unit tests swift file that was created by XCode when I initially added the Test Target (Please save any code you may have created there).

Creating Framework with Tests

Following the steps below we will create a Command Line Interface (CLI) Swift application with Test Target.

1) Add a new Framework Target

2) Name it

preferably name with a Framework suffix for enhanced readability

Please observe that now, we can include Unit tests in our framework

3) Verify we have a Test Target

Now we got the Framework and Tests Targets. Very promising!

4) Add Tests

Now let's add back the tests of our isPalindrome(_:) function to CLI_Unit_Tests_FrameworkTests.swift file.

func testIsPalindrome() {

        let s0 = ""; //false
        let s1 = "a"; // true
        let s2 = "aa"; // true
        let s3 = "abba"; // true
        let s4 = "abcba"; // true
        let s5 = "abc"; // false
        let s6 = "abcd"; // false
        let s7 = "öooooö"; //true
        let s8 = "öooooø"; //false

        XCTAssert(isPalindrome(s0) == false, "Test of \"\(s0)\" is wrong")
        XCTAssert(isPalindrome(s1) == true, "Test of \"\(s1)\" is wrong")
        XCTAssert(isPalindrome(s2) == true, "Test of \"\(s2)\" is wrong")
        XCTAssert(isPalindrome(s3) == true, "Test of \"\(s3)\" is wrong")
        XCTAssert(isPalindrome(s4) == true, "Test of \"\(s4)\" is wrong")
        XCTAssert(isPalindrome(s5) == false, "Test of \"\(s5)\" is wrong")
        XCTAssert(isPalindrome(s6) == false, "Test of \"\(s6)\" is wrong")
        XCTAssert(isPalindrome(s7) == true, "Test of \"\(s7)\" is wrong")
        XCTAssert(isPalindrome(s8) == false, "Test of \"\(s8)\" is wrong")
    }

5) Switch Scheme to Framework

To run tests we will need to change Scheme from the CLI App Target to the Framework (our code and tests now reside in Framework as you can remember from the previous steps)

6) Move code from App to Framework

Cannot find 'isPalindrome' in scope Compilation problem is back with us. Why? We created the Framework and Tests but forgot to move our function from main.swift for the framework!

Let's add a new file and name it utils.swift. Please observe that we're adding the file to

• CLI_Unit_Tests_Framework (the Framework we just created) and
• CLI_Unit_Tests_FrameworkTests (Framework Test Target that XCode automatically created for us) Targets upon creation.

You can always change it later in Build Phases->Compile Sources settings of any given Target.

Next, let's move our isPalindrome(_:) function to utils.swift.

func isPalindrome<S: StringProtocol>(_ str: S) -> Bool {
    guard nil != str.first else { return false }

    var headIdx = str.startIndex
    var tailIdx = str.index(before: str.endIndex)

    while headIdx <= tailIdx {
        if str[headIdx] != str[tailIdx] { return false }

        headIdx = str.index(after: headIdx);
        tailIdx = str.index(before: tailIdx);
    }
    return true;
}

7) Tests are running now

This time our Tests ran and immediately crashed the code on a single-character string!

Let's fix the code by adding if headIdx == tailIdx { break } to isPalindrome(_:) function and run the tests again

(Fixed code)

func isPalindrome<S: StringProtocol>(_ str: S) -> Bool {
    guard nil != str.first else { return false }

    var headIdx = str.startIndex
    var tailIdx = str.index(before: str.endIndex)

    while headIdx <= tailIdx {
        if str[headIdx] != str[tailIdx] { return false }
        if headIdx == tailIdx { break }

        headIdx = str.index(after: headIdx);
        tailIdx = str.index(before: tailIdx);
    }
    return true;
}

7.1) Make Tests pass

This time the Function tested nicely and passed all the tests

Hooray! We did it! (not entirely though)

8) Switch back to App Target

Let's return to our main.swift and try to run the CLI application again.
Don't forget to change the Scheme back to cli_unit_tests application.

Now the CLI app doesn't compile though.

Why that? As you can remember we moved isPalindrome(_:) function to the Framework so we can Test it. Now we need to add the Framework to the CLI application by adding import CLI_Unit_Tests_Framework in main.swift to have access to isPalindrome(_:)

But the problem is still there!

9) Fixing Code Access

We have to do yet two more steps to make isPalindrome(_:) function accessible to the CLI application in cli_unit_tests App Target.

10) Fix Access Levels

We need to make our function public, otherwise it won't be accessible to cli_unit_tests App Target.

public func isPalindrome<S: StringProtocol>(_ str: S) -> Bool

This has to deal with the fact that by default function has internal access level. In practice, that means that code outside of the module where the function is defined cannot see a function defined as internal.

As Swift book puts this:

Any internal implementation details of your framework can still use the default access level of internal, or can be marked as private or file private if you want to hide them from other parts of the framework’s internal code. You need to mark an entity as open or public only if you want it to become part of your framework’s API.

You may ask where did we get the modules? If you recall we moved the isPalindrome(_:) function to a Framework Target which is now a separate module from cli_unit_tests App Target (which is another module).

The Project is now split into 3 modules:

• CLI App Target
• Framework Target
• Framework Test Target

You can read more on modules in Swift book.

A note on @testable

You may have noticed that CLI_Unit_Tests_FrameworkTests is a separate Target from CLI_Unit_Tests_Framework (hence a separate module) but testing the code inside Framework didn't require marking isPalindrome(_:) function as public.

If you carefully look into CLI_Unit_Tests_FrameworkTests.swift file (created automatically by XCode for us) you will see that the import line contains @testable keyword.

@testable import CLI_Unit_Tests_Framework

Swift book has a succinct explanation of it

...a unit test target can access any internal entity, if you mark the import declaration for a product module with the @testable attribute and compile that product module with testing enabled.

Thus our Test Target could access isPalindrome(_:) without it being public.

10) Adding Dependencies (Linking Modules)

In the previous step, we fixed the Access Level of isPalindrome(_:) function but it won't be sufficient for the cli_unit_tests App Target to get access to the function.

If you wonder why that happens - recall that the cli_unit_tests App Target and CLI_Unit_Tests_Framework Framework are separate modules and they don't know about each other despite being inside the same project.
XCode added CLI_Unit_Tests_Framework as a dependency to CLI_Unit_Tests_FrameworkTests automatically upon creation so you didn't notice this problem in the previous steps

You can read more about Managing dependencies in XCode in Apple's official documentation.

To add a dependency you need to

1. Select Project
2. Select App Target
3. Open Build Phases tab
4. Open Dependencies section and
5. press + button

Next, you need to select the CLI_Unit_Tests_Framework Target as a dependency.

Now if you try to Build the cli_unit_tests App Target everything should build and run successfully.

If you peek into the Builds logs you can see how dependencies get compiled before the App target and then there's a Linking step right before the final App is ready for signing.

Hooray! We built and run both Tests and the App!

Summary

Creating CLI Apps in Xcode with a working Test Target requires several steps to make things work which is unfortunate. Hopefully, Apple will fix it sometime.

On the other hand, separating code into Modules provides the added benefit of having clear code boundaries and better abstractions while encouraging code reuse.

Comments

[Shimi Halperin]

Thanks this helped me a lot. I was bloodying my head against trying to get unit tests for a CLI app without using a Framework.