I'm adding the new custom service by command
firewall-cmd --permanent --new-service=ssh2
but can't find how to remove it ...
This is an old question; however, I think this would be useful to share.
When you create a service, a <service-name>.xml file will be created at /etc/firewalld/services/
Now, when you delete a custom service, make sure you delete the corresponding file(s) - as there may be a .xml.old file for that service - too.
firewall-cmd --permanent --remove-service=ssh2
rm -f /etc/firewalld/services/ssh2.xml*
firewall-cmd --reload
Check to see if your service is still available or not:
firewall-cmd --info-service=ssh2
This should return an error. :)