Assuming Alice and Bob are using WebRTC to establish a connection, and there is a TURN server available.
Both Alice and Bob set their iceServers to this server and also set the iceTransportPolicy as "relay".
Strangely, I am unable to establish a connection between the two. My turn server is Coturn. Why?
Your first question is easy: yes, both (and should) set the policy to relay if that's what you want to enforce.
The second question, why does it not work, is difficult to answer without more detail. Perhaps the better question is how to debug this, so that's how I'll try to help. I usually do this by printing all returned candidates to the console of type relay. If either side does not receive any such candidates, then you have a smoking gun (often an authentication issue with coturn).
The other helpful tool is the telnet CLI coturn has (see the /etc/turnserver.conf file for details). Activate that, log in, and use ps to show all current sessions. You should find two active sessions, one for each side.