Failed to access Logic app via APIM when restricting APIM IP in logic app

Problem statement:

I have a logic app which I have exposed via APIM[I want this way as this is my design approach] and I want to allow only specific inbound IP address(in my case APIM IP) to access my logic app.

I have restricted the IPs in apim as follows, and which is working as expected.

    <ip-filter action="allow">
        <address-range from="xx.xx.xx.x1" to="xx.xx.xx.x5" /> //currently this is of no use.
        <address>xx.xx.xx.xx</address> //my local machine ip.
    </ip-filter>

Now, I have allowed the specific ip(apim ip) in my logic app and hit the request from my local machine, logic app prompt me an error.

When I added my local machine's Ip in logic app then it works as expected.

So,I think What is the point of restricting the ips in APIM because I need to allow all the IPs in logic app as well?

could anyone please assist/elaborate me what i am doing wrong.

I have analyzed the issue and below is the finding.

Getting Unauthorized is because of not whitelisting the local machine IP from which I was exploring the logic app to see the requested content.

Everything else is working as expected.

hope this will help!