mortengregersen / bagbutik

Generated Swift code for the App Store Connect API based on the official OpenAPI spec.

Generated Swift code for the App Store Connect API based on the official OpenAPI spec.

What is Bagbutik?

Bag·bu·tik (noun) /ˈb̥a̝w-ˌb̥u-tiːk/ - (Danish) A room behind a store

Bagbutik is two things, a command line tool for generating Swift code, and a library with the generated code for the App Store Connect API. The command line tool decodes the official OpenAPI spec for Apple's App Store Connect API.

While generating the code, the command line tool also downloads the official documentation from Apple's Documentation site for the App Store Connect API and applies it to the generated code, so you can get documentation right in Xcode.

The command line tool is only needed, when Apple publishes a new version of the spec. A GitHub Action automatically checks for new versions of the spec and creates a pull request with the changes.

How to use Bagbutik

Bagbutik uses JSON Web Tokens (JWT) for authorization. You obtain the required keys from your organization's App Store Connect account.

See Creating API Keys for App Store Connect API for how to create your keys.

Here is a basic example for fetching all bundle IDs for iOS apps including related profiles sorted by seed ID and the bundle ID itself descending.

import Bagbutik_Core
import Bagbutik_Provisioning

let service = BagbutikService(jwt: try .init(
    keyId: "P9M252746H",
    issuerId: "82067982-6b3b-4a48-be4f-5b10b373c5f2",
    privateKey: """
    -----BEGIN PRIVATE KEY-----
    MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgevZzL1gdAFr88hb2
    OF/2NxApJCzGCEDdfSp6VQO30hyhRANCAAQRWz+jn65BtOMvdyHKcvjBeBSDZH2r
    1RTwjmYSi9R/zpBnuQ4EiMnCqfMPWiZqB4QdbAd0E7oH50VpuZ1P087G
    -----END PRIVATE KEY-----
    """
))
let response = try await service.request(
    .listBundleIdsV1(filters: [.platform([.iOS])],
                     includes: [.profiles],
                     sorts: [.seedIdDescending, .idDescending])
)
print(response)

How to get Bagbutik into a project

Bagbutik supports Swift Package Manager and the internal libraries used to generate the code is also managed by it.

The library of generated code actually consists of multiple libraries, to let the consumer only depend on the endpoints needed. So to get Bagbutik, you need to decide which functionality you need.

In your Package.swift add Bagbutik as a dependency:

dependencies: [
    .package(url: "https://github.com/MortenGregersen/Bagbutik", from: "8.0.0"),

Then in targets add the libraries needed by your target:

targets: [
    .target(
        name: "Awesome",
        dependencies: [
            .product(name: "Bagbutik-AppStore", package: "Bagbutik"),
            .product(name: "Bagbutik-TestFlight", package: "Bagbutik"),

If all libraries are needed, the umbrella library Bagbutik can be used:

targets: [
    .target(
        name: "Awesome",
        dependencies: [
            .product(name: "Bagbutik", package: "Bagbutik"),

Remember to replace the hyphen with an underscore when importing the libraries.

When importing Bagbutik-TestFlight write:

import Bagbutik_TestFlight

Bagbutik-Core

The core library is Bagbutik-Core which contains the BagbutikService, the JWT, protocols and the general generated types like ErrorResponse and PagingInformation.

All other libraries depend on Bagbutik-Core. This is the only library which is documented on Bagbutik.dev, as the other libraries only contain types which are already documented by Apple.

Bagbutik-Models

Because the models from the API are used from many different endpoints, they are all located in Bagbutik-Models. All libraries with endpoints depend on Bagbutik-Models.

Bagbutik-<API-Area>

The endpoints of all the areas of the App Store Connect API are located in different libraries with a describing name.

Right now there are 7 libraries with endpoints:

• Bagbutik-AppStore: Manage all aspects of your app, App Clips, in-app purchases, and customer reviews in the App Store.
• Bagbutik-GameCenter: Manage Game Center data and configurations for your apps.
• Bagbutik-Marketplaces: Manage keys, packages, and search for the marketplace distribution of your app.
• Bagbutik-Provisioning: Manage the bundle IDs, certificates, devices and provisioning profiles for your app.
• Bagbutik-Reporting: Download your sales and financial reports and get power and performance metrics, logs, and signatures.
• Bagbutik-TestFlight: Manage your beta testing program, including beta testers and groups, apps, App Clips, and builds.
• Bagbutik-Users: Manage users and email invitations to join your App Store Connect team.
• Bagbutik-XcodeCloud: Automate reading Xcode Cloud data, managing workflows, and starting builds.

Manual patches applied to OpenAPI Spec

The OpenAPI Spec provided by Apple do not always align with the data received from the API. Whenever such mismatches are identified, feedback is submitted to Apple and a patch is applied. Whenever an issue is resolved, the patch is removed again.

Currently open feedback and applied patches

FB8977648: The "BundleIdPlatform" schema is missing "UNIVERSAL" and "SERVICES"

• Submitted: January 21st 2021.
• Updated: October 14th 2022 - the type "SERVICES" is also missing.

Title: App Store Connect API is missing the "UNIVERSAL" type for the BundleIdPlatform schema

Description

In the OpenAPI spec for the App Store Connect API the "BundleIdPlatform" schema is said to only be "IOS" or "MAC_OS". This is not right as universal apps (iOS and macOS) has a "UNIVERSAL" platform.

FB12292035: ErrorResponse.Errors has required optional "detail" and no "associatedErrors" in "meta"

• Submitted: June 9th 2023.

In Apple's OpenAPI spec the detail property on ErrorResponse.Errors is marked as required. On 12/1/23 some errors (with status code 409) has been observed, with no detail.

In Apple's OpenAPI spec and documentation the associatedErrors is not mentioned in meta property (last checked 12/1/23). But it is observed when creating a ReviewSubmissionItem with an AppStoreVersion fails.

FB13701181: App Store Connect API Spec is missing "DEVELOPER_ID_APPLICATION_G2" type for the Certificate Type schema

• Submitted: March 28th 2024.

In the OpenAPI spec for the App Store Connect API the “CertificateType” schema is said to not include "DEVELOPER_ID_APPLICATION_G2”. This is not right as “Certificates” endpoints can have a “DEVELOPER_ID_APPLICATION_G2” type.

When creating certificates on the developer portal, it is also possible to select the G2 Sub-CA (which corresponds to “DEVELOPER_ID_APPLICATION_G2”.

Closed feedback (removed patches)

• FB13540097: Almost all of the schemas ending in “WithoutIncludesResponse” has wrong "data" type

  • Submitted: January 14th 2024.
  • Resolved: July 11th 2024 (Spec version 3.5).

• FB9963088: The xcodeMetrics schema has no properties or attributes in the OpenAPI spec

  • Submitted: March 21st 2022.
  • Resolved: July 12th 2022 (Spec version 2.0).

• FB10029609: Required 'links' property missing on included BuildBundle

  • Submitted: May 27th 2022.
  • Resolved: At least since May 23rd 2023.

• FB13071298: App Store Connect API Spec is missing "VISION_OS" type for the Platform schema

  • Submitted: August 28th 2023.
  • Resolved: January 25th 2024 (Spec version 3.2).

• FB13539766: App Store Connect API Spec is missing "APP_APPLE_VISION_PRO" type for the Screenshot Display Type schema

  • Submitted: January 14th 2024.
  • Resolved: January 25th 2024 (Spec version 3.2).

bagbutik's Issues

Is it a @NullCodable issue?

Try create a new version of my app:

func newVersion(appID: String, versionString: String) async -> AppStoreVersionResponse? {
    do {
        let appStoreVersionResponse = try await service.request(
            .createAppStoreVersion(
                requestBody: .init(
                    data: .init(
                        attributes: .init(
                            platform: .iOS,
                            versionString: versionString
                        ),
                        relationships: .init(
                            app: .init(
                                data: .init(
                                    id: appID
                                )
                            )
                        )
                    )
                )
            )
        )

        return appStoreVersionResponse
    } catch {
        print("ServiceError:", (error as? ServiceError)?.description ?? "")
        return nil
    }
}

An unhandled HTTP error occurred:

ServiceError: An unhandled HTTP error occurred. Status code 422. Data as UTF-8 string: {
  "errors" : [ {
    "id" : "0a77804d-9c04-4d71-93b4-a76f52bd33b2",
    "status" : "422",
    "code" : "ENTITY_UNPROCESSABLE",
    "title" : "The request entity is not a valid request document object",
    "detail" : "The data for relationship 'appStoreVersionLocalizations' is missing the required 'type' property."
  }, {
    "id" : "41bf8ba1-6b4d-445d-b985-3d3ca365eced",
    "status" : "422",
    "code" : "ENTITY_UNPROCESSABLE",
    "title" : "The request entity is not a valid request document object",
    "detail" : "The data for relationship 'build' is missing the required 'type' property."
  } ]
}

Maybe it's a @NullCodable related issue.

Release builds are much slower than debug builds.

Bagbutik-Models takes ~0:24 for a debug build and ~2:10 for a release build. The other targets don't have this same difference in build times.

I asked about this a while ago on Mastodon and you mentioned that Codable could be a source of the issue.
https://mastodon.social/@mortengregersen/110210277236346597

Experiment

I had some free time so I did a bit of an experiment to see what the effect of the generated Codable implementation is on the release build time. I also included CaseIterable just incase anything generated by the compiler was the cause. I used https://github.com/sharkdp/hyperfine to benchmark the build times one different branches with the following command.

hyperfine \
  --parameter-list branch main,generate-codable,remove-codable \
  --runs 5 \
  --command-name "Building {branch}"\
  --warmup 1 \
  --setup 'git checkout {branch}' \
  --prepare 'swift package clean' \
  'swift build --configuration release --target Bagbutik-Models'

Baseline

This was run on main (e1bdc2d).

Benchmark 1: Building main
  Time (mean ± σ):     133.187 s ±  2.863 s    [User: 290.170 s, System: 55.831 s]
  Range (min … max):   130.391 s … 137.076 s    5 runs

Generating Codable and CaseIterable Conformances

This was run on https://github.com/CraigSiemens/Bagbutik/tree/generate-codable

On this branch I updated the generation to create the conformances for Codable (including CodingKeys) and CaseIterable for all types in Bagbutik-Models.

Benchmark 2: Building generate-codable
  Time (mean ± σ):     152.551 s ±  3.086 s    [User: 310.417 s, System: 57.085 s]
  Range (min … max):   150.324 s … 157.834 s    5 runs

That seems to make the build slower still, so it seems the slower builds are not caused by the compiler generating the conformance, unless it's still doing checks for whether it needs to add conformance to all the files. Watching CPU usage, it spends most of the time using a single core before switching to using all available cores.

Removing Codable and CaseIterable

This was run on https://github.com/CraigSiemens/Bagbutik/tree/remove-codable

On this branch I modified the generation to remove Codable and CaseIterable from all models to confirm whether it was the source of the slower build times. This was just for experimenting since the change also causes the rest of the project to fail to build/function.

Benchmark 3: Building remove-codable
  Time (mean ± σ):     37.170 s ±  0.244 s    [User: 119.392 s, System: 18.215 s]
  Range (min … max):   36.890 s … 37.482 s    5 runs

The build was much faster confirming that Codable adds a lot to the build time.

Results

Summary
  'Building remove-codable' ran
    3.58 ± 0.08 times faster than 'Building main'
    4.10 ± 0.09 times faster than 'Building generate-codable'

It appears that adding Codable, regardless of whether it uses the compiler generated conformance, is the cause of the increase in build time. The next steps feel like they should be:

• Report this as an issue on the swift compiler as this feels like unexpected behavour. Adding multiple files conforming to Codable shouldn't impact build times that much.
• Investigate how the build time is affected with less files being built. That would help determine whether splitting the models into multiple modules would help the issue.

Debug configuration

Debug builds show a similar relationship between the different branches, though the total time is shorter and relative improvement is less.

Benchmark 1: Building main
  Time (mean ± σ):     22.309 s ±  1.616 s    [User: 126.424 s, System: 12.401 s]
  Range (min … max):   20.469 s … 23.955 s    5 runs
 
Benchmark 2: Building generate-codable
  Time (mean ± σ):     25.719 s ±  1.523 s    [User: 138.862 s, System: 12.123 s]
  Range (min … max):   23.974 s … 28.104 s    5 runs
 
Benchmark 3: Building remove-codable
  Time (mean ± σ):     11.727 s ±  0.755 s    [User: 54.623 s, System: 9.200 s]
  Range (min … max):   10.594 s … 12.276 s    5 runs
  
Summary
  'Building remove-codable' ran
    1.90 ± 0.18 times faster than 'Building main'
    2.19 ± 0.19 times faster than 'Building generate-codable'

Unstable Release 9.1.2

As part of #168 it introduced an "unstable" release for the Zip dependency so if you try to use SPM with that version it will fail with an error similar to this:

I'm not sure what's the best course of action here but it might be a good idea to look into removing the Zip dependency.
A hacky/faster solution would be to spin up a process calling the unzip command using https://developer.apple.com/documentation/foundation/process# since it's only being used for that purpose for the CLI.

I can work on it if this is an acceptable change.

CiScheduledStartCondition.Schedule.days has type Items? instead of [Items]?

Hi Morten,

There's a small issue in the CiScheduledStartCondition.Schedule model for Xcode Cloud. Currently the type of days for a scheduled start is the day itself instead of an array of days as specified in the App Store Connect API.

Bagbutik/Sources/Bagbutik-Models/XcodeCloud/CiScheduledStartCondition.swift

Line 45 in 25f424b

public var days: Items?

where as App Store Connect API would expect something like:

public var days: [String]?
public var days: [Items]? // In Bagbutik's case

Check the API specs for more info: CiScheduledStartCondition.Schedule

Reduce file size of framework

As mentioned in DMs, Bagbutik currently makes apps very large due to a lot of the dependencies.

Illegal Parameter "limit"

Environment
Bagbutik Version: 10.3.0
Platform: macOS Sonoma 14.5

Details
I'm making the following request to get data related to an experiment. I noticed that if I make the request without the limit parameter, I only get 10 localizations (in a random order). This isn't great for apps that have many localizations in the experiment.

let treatmentID = treatmentRelationship.id
let response = try await service.request(
    .getAppStoreVersionExperimentTreatmentV1(
        id: treatmentID,
        includes: [.appStoreVersionExperimentTreatmentLocalizations],
        limit: 50
    )
)

When I make the request, I get the following error:

badRequest(Bagbutik_Core.ErrorResponse(errors: Optional([Bagbutik_Core.ErrorResponse.Errors(code: "PARAMETER_ERROR.ILLEGAL", detail: Optional("The parameter 'limit' can not be used with this request"), id: Optional("020add9f-38c7-41c8-a69c-9e8fba508db0"), meta: nil, source: Optional(Bagbutik_Core.ErrorResponse.Errors.Source.jsonPointer(Bagbutik_Core.JsonPointer(pointer: nil))), status: "400", title: "A given parameter is not allowed for this request")])))

Is something misaligned in the spec? Or is this a bug in the ASC API?