Hello Swift Testing, Goodbye XCTest

Meet Apple’s new testing framework and see the main differences from XCTest, with real examples

Introduction

Starting in WWDC24, Apple officially announced the new Swift Testing framework with a lot of advantages over our current XCTest. I took my time in the project I am currently working on to refactor some of my tests.

This article shows the differences between XCTest and Swift Testing and some of the common scenarios you probably will face, like injecting your Spies or adding your SUT object.

Getting Started & Key Differences

Good to know

Before jumping straight to code, it is good to know:

• Swift Testing runs tests in parallel by default, for sync and async tests, while XCTest only supports parallelization using multiple processes each running one test at a time.
• If your XCTests are not prepared to be run in parallel prepare to add .serialized trait everywhere
• Swift Testing runs randomized tests by default.

Starting with the Basics

With the current XCTest approach, you need to import the XCTest framework, create a class that inherits from XCTestCase and create test functions that begin with the word test:

import XCTest

final class JobOfferViewModelTests: XCTestCase {
    func test_receiveJobOffer_givenSuccess_shouldDisplayAlert() {
      ...
    }
}

With the new Swift Testing framework, on the other hand, you need to import the Testing library, and declare your testing function using @Test macro:

import Testing

@Test receiveJobOffer() {
   ...
}

Simple testing function

Use #expect(...) to do the assertions.

import Testing 

@Test myFirstTest() {
  #expect(1 == 1) 
}

All of the old XCTestCaseassertions below were replaced by the new #expect function

// XCTest
XCTAssertTrue()
XCTAssertFalse()
XCTAssertEqual()
XCTAssertNil()
XCTAssertNotNil()
XCTAssertLessThan()
XCTAssertGreaterThan()
XCTAssertLessThanOrEqual()
XCTAssertGreaterThanOrEqual()
...

// Swift Testing
#expect()

Organizing functions with Suite type

As you can see, your tests run in a single file without a class, but when working with a large selection of test functions, organizing them into test suites can be helpful.

Per Apple documentation, a test function can be added to a test suite in one of two ways:

• By placing it in a Swift type.
• By placing it in a Swift type and annotating that type with the @Suite attribute

You can use struct, final class, enum or actor as your test suite:

@Suite struct JobOfferViewModelTests {
  @Test func receiveJobOffer() { ... }
}

@Suite final class JobOfferViewModelTests {
  @Test func receiveJobOffer() { ... }
}

@Suite enum JobOfferViewModelTests {
  @Test func receiveJobOffer() { ... }
}

@Suite actor JobOfferViewModelTests {
  @Test func receiveJobOffer() { ... }
}

@Suite type can be omitted because the compiler is able to identify if your struct contains @Test functions, but you can use it to allow customization.

struct JobOfferViewModelTests { // Works ✅
  @Test func receiveJobOffer() { ... }
}

@Suite("Job Offer Success Tests") struct JobOfferViewModelTests { // Works ✅
  @Test func receiveJobOffer() { ... }
}

@Suite("Job Offer Tests") struct JobOfferViewModelTests { // Works ✅
  @Suite("Job Offer - Success") struct Success {
    @Test func receiveJobOffer() { ... }
  } 
  @Suite("Job Offer - failure") struct Failure {
    @Test func receiveJobOffer() { ... }
    @Test func declinedJobOffer() { ... }
  }
}

When dealing with @Suites, you cannot initialize your properties outside and use them in the nested suites. The following code does not work:

@Suite("Job Offer Tests") struct JobOfferViewModelTests { // Does not work ❌
  let getJobOfferUseCaseSpy = GetJobOfferUseCaseSpy()
  let sut = makeSUT()

  @Suite struct Success {
    @Test func receiveJobOffer() {
       sut.listenToJobOfferUpdates() // Error here: Instance sut cannot be used
       #expect(getJobOfferUseCaseSpy.receiveJobOfferCalled == true)
    }
  }
}

You should create the properties within the suites, and duplicate between all of them:

@Suite("Job Offer Tests") struct JobOfferViewModelTests { // Works ✅
  @Suite struct Success {

    // Create the spies and sut for "Success" suite
    let getJobOfferUseCaseSpy = GetJobOfferUseCaseSpy()
    let sut = makeSUT()

    @Test func receiveJobOffer() {
       sut.listenToJobOfferUpdates()
       #expect(getJobOfferUseCaseSpy.receiveJobOfferCalled == true)
    }
  }

  @Suite struct Failure {

    // Create the spies and sut again, but for "Failure" suite
    let getJobOfferUseCaseSpy = GetJobOfferUseCaseSpy()
    let sut = makeSUT()

    @Test func failedToGetJobOffer() { ... }
  }
}

setUp and tearDown

In XCTestCase you can configure properties before and after each test run by using the available methods:

func setUp() { 
  // Before testing function, configure something...
}

func tearDown() {
  // After testing function, reset values to default state...
}

With Swift Testing these methods do not exist. Use init and deinit instead. For struct, memberwise initializer will be used.

For each test case, a new instance of your struct, class, or any other object will be created and the state will never be shared across tests.

Put Everything Together

Putting the basic concepts together, here is an example of a simple ViewModel that validates if a function opens a URL using the correct passenger contact information:

import Testing

final class JobOfferViewModelTests {
  private let applicationSpy = ApplicationSpy()
  private lazy var sut: JobOfferViewModel = {
    // Implementation hidden for brevity
  }()
    
   @Test callPassenger_givenPassengerNumber_shouldOpenCorrectURL() {
      sut.jobDataToBeReturned = .fixture(passengerContact: "1234-5678")
      sut.callPassenger()
      #expect(applicationSpy.openedURL == URL(string: "tel://1234-5678"))
   }
}

Note that this function uses concepts such as .fixture() and Test Doubles. If you're not familiar with these terms, please check my Unit Tests 101 Article!

iOS Unit Test Master 101 — Vol. 1

As you can see, the test name is a little bit long. Instead of using that naming convention, the new @Test property wrapper introduces a new way to identify your tests with Traits. Let's take a look!

Traits

XCTest does not support traits.

Traits add descriptive information about a test, can be customized whether a test runs, and modify how a test behaves. Let's meet some of them:

Description trait

To give more details about your test. Xcode will show your description instead of the function name.

@Test("Given correct user number and password, should return success")
func validateLogin() {...}

Tag trait

A type representing a tag that can be applied to a test. We can create new tags by extending the Tag object and applying the @Tag macro:

@Test(.tags(.jobBidding))

private extension Tag {
  @Tag static var jobBidding: Self
}

Tags provide semantic information for a test that can be shared with any number of other tests across test suites, source files, and even test targets. They can be filtered in the left-side menu, run separately from the other cases, or even filtered from the Insights menu after a test run.

Benefits of using tags:

• Helps you analyze results across test targets
• Quickly include or exclude tags from your test plan
• See insights about each tag

Bug trait

Receive the bug URL and some description to keep track of your bugs using tools like Jira.

@Test(.bug("https://atlassian.net/jira/software/ZM/123", "Fix something..."))
func callPassenger() {...}

Toggle trait

Enable or disable your test based on conditions. By doing this, you specify a runtime-evaluated condition that skips your test if the condition is met:

@Test(
.disabled("This test is crashing, please fix") // Works ✅
.bug("https://atlassian.net/jira/software/ZM/123", "Fix failing unit test")
)
func callPassenger() {}

@Test(.disabled()) // Works ✅
func routeToPaymentScreen() {}

// Or 

@Test(.enabled(if: FeatureFlag.isCallPassengerEnabled)) // Works ✅
func callPassenger() {}

Given a scenario where you know a unit test is failing, instead of commenting out or disabling using the .disabled() trait, prefer using the withKnownIssue.

@Test func thisFunctionShouldFail() async {
    withKnownIssue {
      sut.thisFunctionShouldFail()
      #expect(routeTriggerSpy.values == [.pop(animated: false)])
   }
}

The failing test will be recorded and all tests will pass. When you fix this breaking unit test, Xcode will notify you so you can remove the withKnownIssue.

This function is the equivalent of XCTest’s XCTExpectFailure

TimeLimit trait

Set a maximum time limit for a test:

@Test(.timeLimit(.minutes(3))) 
func callPassenger() {}

@Test(.timeLimit(.seconds(1))) 
func callPassenger() {}

⚠️ Careful! TimeLimitTrait.Duration is only available for iOS 16.0, macOS 13.0, tvOS 16.0, and watchOS 9.0 or newer.

Parametrized Tests

Back in XCTestCase, to run multiple scenarios for a test we could create multiple unit test functions:

import XCTest

func test_paymentDescription_givenErp_shouldReturnCorrectValue() {...}
func test_paymentDescription_givenExtraDistance_shouldReturnCorrectValue() {...}
func test_paymentDescription_givenExtraStop_shouldReturnCorrectValue() {...}

As your test cases grow, you will need to create more and more test functions that will probably test the same thing but with different parameters. To improve that, we could use forEach:

import XCTest

func test_descriptionString_shouldDisplayCorrectlyWithPaymentType() {
  let paymentType: [PaymentType] = [
      .creditCard,
      .payNow,
      .nets,
      .onBoard
  ]
  paymentType.forEach { paymentType in
    let sut = PaymentAdditionalItemViewModel(
        payment: .fixture(type: paymentType)
    )
    XCTAssertEqual(sut.payment.description, paymentType.description)
  }
}

Much better, but how to improve? Swift Testing introduced Parametrized Tests, which are basically tests that take one or more parameters. When the test function runs, Swift Testing automatically splits it up into separate test cases, one per argument. To use it, just pass the arguments: and add the specific parameter to the function:

@Test(
  "Given different payment type, should display correct alert",
  arguments: [PaymentType.creditCard, .payNow, .nets, .onBoard] // Arguments
)
func displayAlertMessage(paymentType: PaymentType) { // Function that receives the argument
  let sut = PaymentAdditionalItemViewModel(
        payment: .fixture(type: paymentType)
    )  
  #expect(sut.payment.description == paymentType.description)
}

Arguments are independent of each other, meaning that you can re-run individual test cases, and you can run them in parallel as well.

Any sendable collection, including arrays, dictionaries, ranges, and OptionSet are valid to be used as a test attribute.

You can also pass arguments of different types. The function below is receiving (TaxiRideType, Driver) as argument:

@Test(arguments: [
  (TaxiRideType.metered, Driver.fixture(name: "Joel")),
  (TaxiRideType.normal, Driver.fixture(name: "Chong"))
])

⚠️ Careful! For multiple arguments, Swift Testing will run your testing function for every combination of elements, which may increase the amount of testing for each new item added to one of the parameters.

You can use .zip to produce a sequence of tuples instead of pairing them up.

@Test(arguments: [
  .zip(TaxiRideType.metered, Driver.fixture(name: "Joel")),
  .zip(TaxiRideType.normal, Driver.fixture(name: "Chong"))
])

You can even combine Parametrized Tests + Description Trait. Use the new CustomTestStringConvertible protocol with testDescription property to generate custom descriptions for each of your arguments:

enum PaymentType: String, CustomTestStringConvertible {
    case creditCard, debitCard, payNow, onBoard
    
    var testDescription: String {
        "Payment of type \(rawValue) used"
    }
}

Scenario — Passing Result, Error type as arguments

If you want to test Result type with success and failure, you can also pass a custom Array type to the arguments:

@Test(
  "Tap Bid button with Success and Failure",
  .tags(.jobBidding),
  arguments: Array<Result<Void, ErrorEnvelope>>([.success(()), .failure(.default)])
)
func onTapBidButton(getJobExpectedResult: Result<Void, ErrorEnvelope>) async {
  jobServiceSpy.getJobExpectedResult = getJobExpectedResult
  await sut.onTapBidButton()
  #expect(jobServiceSpy.jobNo == "12345")
  #expect(routeTriggerSpy.values == [...])
}

More Examples

Testing Async Code

When writing concurrent test code, you can use the same concurrency features in Swift as you would in production code. Await works exactly the same way and will suspend a test allowing other test code to keep the CPU busy while work is pending.

Mark the test function as async and call your await function:

@Test func fetchJobDetails() async { // Mark function as async
    jobServiceSpy.getJobExpectedResult = getJobExpectedResult
    await sut.onTapBidButton() // Await for the function
    #expect(jobServiceSpy.jobNo == "12345")
    #expect(routeTriggerSpy.values == [...])
}

Validating Error is Thrown

To validate throwing functions, avoid using do-catch

@Test("When trying to pay without internet, should display noInternet error") 
func payment() throws {
  do {
    try sut.doPaymentUseCase()
  } catch {
    #expect(((error as? ErrorEnvelope) == .noInternet))
  }
} 

Prefer using #expect(throws:) instead:

@Test("When trying to pay without internet, should display noInternet error") 
func payment() throws {
  #expect(throws: ErrorEnvelope.noInternet.self) {
    try sut.doPaymentUseCase()
  }       
}

Validating error is NOT thrown

If you need to record a thrown error as an issue without stopping the test function, compare the error to Never:

@Test("When trying to pay with internet, should not throw errors") 
func payment() throws {
  #expect(throws: Never.self) {
    try sut.doPaymentUseCase()
  }       
}

Testing how many times a function was called

In the following scenario, I want the function to be called only once. Use the confirmation function to create a Confirmation for the expected event. In the trailing closure parameter, call the code under test.

The Confirmation object has a built-in counter that can be used by passing the expectedCount parameter.

Since I want to validate how many times my Service is called, I created a serviceCounterHandler closure that is triggered every time a specific function of Spy is called. This property is now receiving the incrementCounter() function, which increments the built-in counter under the hood.

@Test func fetchJobDetails() async {
  jobServiceSpy.serviceToBeReturned = .success(.fixture(jobNo: "0"))

  await confirmation(expectedCount: 1) { incrementCounter in
     jobServiceSpy.serviceCounterHandler = { incrementCounter() } // Trigger this counter every time the service is called 
     await sut.fetchJobDetails()
  }

  #expect(jobServiceSpy.jobDetails.jobNo == "12345")
  #expect(sut.jobData?.jobNo == "0")
}

This is what happens when the function does not meet the count expectation:

On the other hand, you can set the expectedCount to 0 if you want to ensure that a function is not being called at all.

Dealing with Optionals

On XCTestCase, we have a couple of options to deal with optionals.

The first of them is using guard together with XCTFailto force the test to fail immediately and unconditionally.

func test_uploadTrip_shouldReceiveCorrectJobNumber() { 
  guard let tripData = sut.uploadTripData else {
    XCTFail("uploadTripData not available")
  } 
  XCTAssertEqual(tripData.jobNo, "12345")
}

XCTest also has a function XCTUnwrap() that tests if an optional value is nil and throws an error if it is.

func test_uploadTrip_shouldReceiveCorrectJobNumber() throws { 
  let tripData = try XCTUnwrap(sut.uploadTripData)
  XCTAssertEqual(tripData.jobNo, "12345")
}

Using Swift Testing:

• XCTFail was replaced byIssue.record()
• XCTUnwrap was replaced by #require

// Avoid using this 
@Test uploadTrip() { 
  guard let tripData = sut.uploadTripData else {
    Issue.record("uploadTripData not available")
  } 
  #expect(tripData.jobNo == "12345")
}

// Prefer using this
@Test uploadTrip() { 
  let tripData = try #require(sut.uploadTripData)
  #expect(tripData.jobNo == "12345")
}

What’s next

This is just the beginning of a lot of changes in the iOS unit testing community.

Stay tuned for the next articles about Migrating to Swift Testing and Advanced Swift Testing with real examples.

References

• Go further with Swift Testing — WWDC24 Sessions
• Swift Testing Apple Documentation