Cant compile xcframework that depends on 3rd party xcframeworks

I'm currently trying to compile a swift package (which I have wrapped in an xcodeproj specifically for the purpose of being able to release the package as a binary swift package and still run the local code in an example app) into a xcframework so that I can release it as a Cocoapod as well. The project builds fine and runs in the example app in the same repository, so I'm fairly confident in the code itself. However, I'm running into problems while archiving; the classes that come from the dependencies the swift package has can't be found. Both deps (there are only 2) are 3rd party binary swift packages.

▸ Compiling MySegmentLabel.swift

❌  /Users/***/Sources/Views/MySegmentLabel.swift:4:39: cannot find type 'SegmentedControlSegment' in scope

public final class MySegmentLabel: SegmentedControlSegment {
                                    ^~~~~~~~~~~~~~~~~~~~~~~

** ARCHIVE FAILED **

In the above sample output, SegmentedControlSegment is a class from one of the 3rd party binary xcframework dependencies.

I get the feeling that this is a problem with either my archiving command or my xcodeproj build settings, but I just can't figure out which ones I need to tweak or in what way.

In my xcodeproj build settings I have set SKIP_INSTALL=NO and BUILD_LIBRARY_FOR_DISTRIBUTION=YES as Apple recommends (explicitly adding them to the archive command doesnt help either). And this is the command I'm using to archive my xcodeproj:

# Archives the target scheme passed-in to the script
function archive () {
    local sdk=$1
    local configuration=$2
    local build_path="$build_root/$sdk"
    local xcarchive_path="$archive_root/$SCHEME-$sdk.xcarchive"

    xcodebuild \
        -workspace $xcworkspace \
        -scheme "$SCHEME" \
        -configuration $configuration \
        -archivePath $xcarchive_path \
        -derivedDataPath $build_path \
        -sdk $sdk \
        -scmProvider system \
        -showBuildTimingSummary \
        archive | xcpretty

}

archive "iphonesimulator" "Debug" # specifically this one fails. It is able to archive for non-simulators
archive "iphoneos" "Release"

Since it only doesnt compile for the iphonesimulator target, I almost think that the 3rd party libraries maybe didnt bundle the x86_64 frameworks into their xcframeworks (except that can't be because the example app runs fine on simulator when I run the xcodeproj).

In my case (which is probably a rare one) the issue turned out to be that the 3rd party xcframework deps had both been compiled with the setting to exclude the arm64 arch for iphonesimulators. So, when I was trying to archive my SDK for the iphonesimulator sdk for all standard archs (which includes arm64 now due to M1), it would fail to find the class references from the 3rd party deps when it got to archiving for arm64.

Ideally the 3rd party deps would fix their hack to exclude arm64 for simulators, but I dont control those SDKs, so I just had to kick those dependencies down the chain, so the end client (which is an app I'll never need to archive) now has to depend on those 3rd party libs and my swift package needs to be refactored to not need direct access to those 3rd party deps.