I am using this example https://github.com/libp2p/js-libp2p/tree/master/examples/libp2p-in-the-browser and try to connect to a nodejs peer through the given hosted rendezserver ‘/dns4/wrtc-star2.sjc.dwebops.pub/tcp/443/wss/p2p-webrtc-star’ . The peer get found but the connection is not established. If i run the nodejs peer in the same machine as the browser peer they get connected but if the nodejs peer is on a different machine/network the peers find each other but they don’t get connected. Is this normal ?

Yes that's totally normal. WebRTC will only work if you can get through the NAT. IPv6 really helps with NAT traversal too, but it can be trickier with IPv4. So it will work over more than locally, but each peer must be able to directly reach each-other.

https://docs.ipfs.io/how-to/create-simple-chat-app/

We can use WebRTC-Star nodes to help discover other peers we can connect with directly browser-to-browser. If you're familiar with the concept of STUN, it might be helpful to think of them as conceptually similar. Effectively, each connecting node will be given a WebRTC-Star multiaddress that other nodes can use to discover and connect to your browser directly. This means that if you peer with someone using the star node, and the star node goes offline, you remain connected!

[...]

Tip

This is a very clean and effective method of P2P communications; however, sometimes NATs get in the way. We use p2p-circuit to get around that.