SwiftUI ColorPicker with binding to hex string causes large drift

The codebase I am working with (https://github.com/utmapp/UTM) stores configuration as a plist, so a Color value can't be saved as-is.

The method I'm trying to implement now is by having extension methods on NSColor to convert to/from a hex string, and wrapping that in a Binding for a SwiftUI ColorPicker.

The problem is that when changing the selected color in the picker's wheel, there is quite a lot of "drift" (the brightness will move when I change the color, and the color moves seemingly randomly when I move the brightness back up).

Here's the code I'm using for testing (meant to be put in an Xcode Swift Playground):

import SwiftUI
import PlaygroundSupport

struct ContentView: View {
    @State private var colorHex = "#FFFFFF"
    
    private var colorBinding: Binding<Color> {
        Binding<Color> {
            Color(NSColor(hexString: colorHex)!)
        } set: { newValue in
            colorHex = NSColor(newValue).hexString!
        }
    }
    
    var body: some View {
        ZStack {
            colorBinding.wrappedValue
            
            ColorPicker("Pick a color", selection: colorBinding, supportsOpacity: true).padding()
        }
    }
}


extension NSColor {
    var hexString: String? {
        guard let rgbColor = self.usingColorSpace(.sRGB) else {
            return nil
        }
        let red = Int(rgbColor.redComponent * 255)
        let green = Int(rgbColor.greenComponent * 255)
        let blue = Int(rgbColor.blueComponent * 255)
        return String(format: "#%02X%02X%02X", red, green, blue)
    }
    
    convenience init?(hexString hex: String) {
        if hex.count != 7 { // The '#' included
            return nil
        }
            
        let hexColor = String(hex.dropFirst())
        
        let scanner = Scanner(string: hexColor)
        var hexNumber: UInt64 = 0
        
        if !scanner.scanHexInt64(&hexNumber) {
            return nil
        }
        
        let r = CGFloat((hexNumber & 0xff0000) >> 16) / 255
        let g = CGFloat((hexNumber & 0x00ff00) >> 8) / 255
        let b = CGFloat(hexNumber & 0x0000ff) / 255
        
        self.init(srgbRed: r, green: g, blue: b, alpha: 1)
    }
}

PlaygroundPage.current.setLiveView(ContentView())

I was having exact the same problem and was able to solve it by changing the colorspace in the Color initializer from .sGRB to .displayP3.

The hint for me was that the ColorPicker at the slider Tab was showing something like "Display P3 Hex-Colorcode".