I set up a custom domain with App Engine after following the instructions on DNS records:
https://cloud.google.com/appengine/docs/standard/java/mapping-custom-domains
It's working for the default service using URL http://MY_CUSTOM_DOMAIN but I'm unable to access other services with URLs http://SERVICE_ID.MY_CUSTOM_DOMAIN. Do I need to map a different set of DNS records?
Check how to use subdomains and how wildcard mapping works in the GCP docs here (actually these are the next steps at the website which you've followed and linked). Make sure that your DNS provider permits wildcards in CNAME
host entries, otherwise such mapping will not be possible.
This blog post shows a real-life example of matching subdomains to services with App Engine.