elasticsearch 2 node cluster: proper setup?

I have two nodes, but they are not on the same subnet.

Questions:

1) As I understand it, in a 2 node cluster, both should be set to master?

2) The config below is the right thing to do to let the nodes find each other? (Since nodes not on same subnet)

3) Client apps can attach to either node, and do reads and writes?

4) Is the proposed config below correct? (Can I specify "node.master: true" in both configs? Will this make the discovery happen?)

Proposed config:

Node 1:

    cluster.name: mycluster
    node.name: "node1"
    node.master: true
    node.data: true
    discovery.zen.ping.multicast.enabled: false
    discovery.zen.ping.unicast.hosts: ["192.168.100.103"]  # IP of node2

Node 2:

    cluster.name: mycluster
    node.name: "node2"
    node.master: true
    node.data: true
    discovery.zen.ping.multicast.enabled: false
    discovery.zen.ping.unicast.hosts: ["192.168.101.103"]   #IP of node1

2018 UPDATE:

We only use 3 and 5 node clusters now.

Solution

Q1: The ideal number of master nodes to prevent a split brain situation is to have (N/2) + 1 masters, so in your situation, since N=2 the number of masters is 2 as well. Note, though, that having two master nodes is not ideal because it can lead to split-brain situations

Q2: The configs are correct, though you don't need to specify node.master: true and node.data: true as both are true by default.

Q3: That's correct

Q4: Also correct.

Finally, the best way to find out is to run your nodes with those configs and see how it behaves.

You start node1, check the logs and see that node1 is master (since it's the only node)
then you start node2, check the logs and see that node2 joins the cluster
then you bring node1 down, check the logs and verify node2 becomes the master
then you bring node1 up again and verify it joins the cluster
then you bring node2 down and verify node1 becomes the master again.
etc...