SwiftUI: Add ClearButton to TextField

I am trying to add a ClearButton to TextField in SwiftUI when the particular TextField is selected.

The closest I got was creating a ClearButton ViewModifier and adding it to the TextField using .modifer()

The only problem is ClearButton is permanent and does not disappear when TextField is deselected

TextField("Some Text" , text: $someBinding).modifier(ClearButton(text: $someBinding))

struct ClearButton: ViewModifier {
    @Binding var text: String

    public func body(content: Content) -> some View {
        HStack {
            content
            Button(action: {
                self.text = ""
            }) {
                Image(systemName: "multiply.circle.fill")
                    .foregroundColor(.secondary)
            }
        }
    }
}

Use ZStack to position the clear button appear inside the TextField.

TextField("Some Text" , text: $someBinding).modifier(ClearButton(text: $someBinding))

struct ClearButton: ViewModifier
{
    @Binding var text: String
    
    public func body(content: Content) -> some View
    {
        ZStack(alignment: .trailing)
        {
            content
            
            if !text.isEmpty
            {
                Button(action:
                {
                    self.text = ""
                })
                {
                    Image(systemName: "delete.left")
                        .foregroundColor(Color(UIColor.opaqueSeparator))
                        .padding(.trailing, 8)
                        .padding(.vertical, 8)
                        .contentShape(Rectangle())
                }
            }
        }
    }
}