How to remove internal padding to the right of SwiftUI Picker

I'm curious, have you noticed how Pickers have different internal padding when in a Form vs not? Of course, there are many controls that have special looks within the context of a Form. But I would like to have a Picker that isn't in a Form, and I'm trying to line up the Picker button with other controls, and its internal padding is getting in the way.

Here's some code to illustrate:

enum ColorStyle: CaseIterable, Identifiable, CustomStringConvertible {
    case rgb, hsv
    
    var id: Self { self }        // to make it Identifiable
    var description: String {    // to make it CustomStringConvertible
        switch self {
        case .rgb:
            return "RGB"
        case .hsv:
            return "HSV"
        }
    }
}

struct FormComparison: View {
    @State private var colorStyle = ColorStyle.rgb
    
    var colorSource: some View {
        HStack {
            Text("Color Type")
            Spacer()
            Picker("Color Type", selection: $colorStyle) {
                ForEach(ColorStyle.allCases) { style in
                    Text(String(describing: style))
                }
            }
            .pickerStyle(.menu)
            .labelsHidden()
            .border(.green)
        }
    }
    var colorSwatch: some View {
        HStack {
            Text("Color")
            Spacer()
            Color.red.frame(width: 100, height: 25)
        }
    }
    
    var body: some View {
        VStack {
            Form {
                colorSwatch
                colorSource
            }
            
            VStack {
                colorSwatch
                colorSource
                Spacer()
            }
            .padding()
            .background(.blue.opacity(0.15))
            .padding()
        }
        .frame(height: 400)
    }
}

Here's the result:

In the bottom version, which is not in a Form, the Picker is adding some padding to the right of the selector button (clearly visible from the green border I've added), and that's making the non-Form UI look janky.

You can put it in a Menu with a custom label:

Menu {
    Picker("Color Type", selection: $colorStyle) {
        ForEach(ColorStyle.allCases) { style in
            Text(String(describing: style))
        }
    }
} label: {
    HStack(spacing: 5) {
        Text(colorStyle.description)
        Image(systemName: "chevron.up.chevron.down")
    }
}
.labelsHidden()
.border(.green)