Nested Custom Unmarshalling

I need to implement a custom unmarshal logic (to VerifyingKey field) depending on a variable in a field value in upper levels (Type field). I can exemplify the situation as below (originating from Unmarshal remaining JSON after performing custom unmarshalling)

type SingleKey struct {
  Type          string          `json:"type"`
  Others OtherFields   `json:"wrapped_fields"`
}

type OtherFields struct {
  FQDN         string          `json:"fqdn"`
  Address      string          `json:"address"`
  Nonce        int64           `json:"nonce"`
  Challenge    []byte          `json:"challenge"`
  NetworkID    string          `json:"network_id"`
  VerifyingKey []PublicKey     `json:"verifying_key"`
  Signature    []byte          `json:"signature"`
}

// Interface for Public Key
type PublicKey interface {
  // Methods specific to public key functionality
}

// Example concrete implementations (replace with your actual types)
type RsaPublicKey struct {
  PublicKey
  // ... (fields specific to RSA public key)
}

type EcdsaPublicKey struct {
  PublicKey
  // ... (fields specific to ECDSA public key)
}

I have tried many alternatives however, I could not reach to a solution.

Implement a custom unmarshaler for SingleKey:

type singleKeyMarshal struct {
  Type          string          `json:"type"`
  Others      otherFieldsMarshal `json:"wrapped_fields"`
}

type otherFieldsMarshal struct {
  FQDN         string          `json:"fqdn"`
  Address      string          `json:"address"`
  Nonce        int64           `json:"nonce"`
  Challenge    []byte          `json:"challenge"`
  NetworkID    string          `json:"network_id"`
  VerifyingKey []json.RawMessage     `json:"verifying_key"`
  Signature    []byte          `json:"signature"`
}

func (s *SingleKey) UnmarshalJSON(in []byte) error {

  var sm singleKeyMarshal
  if  err:=json.Unmarshal(in,&sm); err!=nil {
    return err
  }
  // Copy all fields from sm to s
  switch sm.Type {
     // Deal with unmarshaling the public keys
     case "RSA": 
       // This depends on how it is marshaled. If it is rsa.PublicKey:
       key:=[]rsa.PublicKey{}
       if err:=json.Unmarshal(sm.Others.VerifyingKey,&key); err!=nil {
           return err
       }
       s.Others.VerifyingKey=key
       
  }
}