How to setup cartridge applications replication between few Tarantool routers on different phisical/virtual servers?
Same as any other nodes - you should press Join existing replicaset
instead of Create new replicaset
when configuring a new router in admin UI.
If you' experiencing any particular issues, please specify them too.
BTW, the screenshot you posted, seems to contain the answer already.