Is it possible to remote-connect to an Azure App Service running on Windows with my IIS Manager?
This seems to have been possible according to this blog post by clicking in IIS Manager File -> Connect to a site.
I refer to the steps mentioned in the link shared by you and in the last step I got an error below.
I did some research on it and it looks like managing the App Service from IIS is not supported anymore. I have not got any official link but the below links might give you some information on it.