SwiftQA Random

How do you detect performance regressions?

1. The core principle: you need a baseline before you can detect a regression

A regression is, definitionally, “worse than before” — which means you need continuous, comparable measurement over time, not a one-off profiling session.

2. XCTest performance tests — the CI-integrated baseline mechanism

func testProfileScreenLoadPerformance() {
    measure(metrics: [XCTClockMetric(), XCTCPUMetric(), XCTMemoryMetric()]) {
        _ = ProfileViewModel(repository: MockRepository())
    }
}

Xcode stores a baseline per test/device configuration and flags a failure if a run deviates beyond a configured threshold. Running these in CI catches regressions at the commit that introduced them. Caveat: CI runners have variable performance characteristics, so these tests are prone to flakiness on shared infrastructure — many teams run them on dedicated hardware or treat them as nightly/directional rather than blocking every PR.

3. Launch time tracking specifically

  • MetricKit’s MXAppLaunchMetric — real-world, aggregated launch time data from actual user devices.
  • Custom os_signpost instrumentation around specific launch stages, reported to your analytics pipeline.

4. Frame rate / hitch tracking for scroll and animation performance

MetricKit’s MXAnimationMetric (hitch time ratio) gives production-level visibility into dropped frames across your real user base.

5. Custom telemetry for app-specific critical paths

let start = ContinuousClock.now
await loadFeed()
let duration = ContinuousClock.now - start
Analytics.log("feed_load_duration", properties: ["ms": duration.milliseconds, "app_version": appVersion])

Dashboards on these metrics, sliced by app version, let you see a regression appear the moment a new version ships.

6. Version-over-version correlation — the actual detection mechanism in practice

Slice every performance metric by app version (and ideally feature flag) so a regression introduced in a specific release is visible as a step-change right at that version’s rollout.

7. Staged rollouts as a built-in regression safety net

If your release process supports phased rollout, performance telemetry from the early cohort acts as an early-warning system — a regression showing up in the 1% cohort is a signal to halt the rollout.

8. Manual/scripted Instruments comparison for pre-release validation

For high-stakes releases, run a scripted Instruments pass comparing the release candidate against the previous shipped version — useful for flows too complex to easily automate as an XCTMetric test.

9. Code review and PR-level awareness for obvious regression risks

Not every regression needs tooling to catch — a PR adding a new synchronous dependency to app launch, or a new heavy computation added to a hot path, is worth a reviewer flagging before merge.

The layered detection strategy, end to end

LayerMechanismCatches
CI, per-PR/nightlyXCTest performance tests (XCTMetric) with stored baselinesRegressions at the commit that introduced them
Production, aggregateMetricKit (launch time, hitch rate, memory)Real-world regressions across genuine device/OS diversity
Production, app-specificCustom telemetry on critical flows, sliced by versionRegressions in flows unique to your app’s business logic
Rollout-timeStaged rollout + early-cohort telemetryRegressions before they reach 100% of users
Pre-release, manualScripted Instruments comparison vs. previous versionComplex/interactive flows hard to automate
Review-timeReviewer awareness of hot-path/launch-path changesObvious risk before it ever reaches any of the above

One-liner if pressed: “Regression detection needs a baseline to compare against, so I’d combine XCTest performance tests with stored baselines running in CI to catch regressions at the commit that introduced them, MetricKit for real-world launch time and hitch-rate data across actual user devices in production, and custom telemetry on the app’s specific critical flows — all sliced by app version so a regression shows up as a visible step-change right at the release that caused it, ideally caught early via a staged rollout before it reaches every user.”