akka.cluster and persistence delivering issue

In our cluster we have four nodes composite of:

• 2 seed nodes (backend)
• 1 worker
• 1 webapi on IIS

The cluster is joined, up and running; when i send a POST to the webapi,:

• IIS join the cluster
• the API recieve the a post and send the message with a Tell
• the message is processed two or three times!
• is happens only for the message sent when IIS join, the following messages are working fine

Here is my IIS config:

<akka>
  <hocon>
    <![CDATA[
            akka.loglevel = INFO
            akka.log-config-on-start = off
            akka.stdout-loglevel = INFO
            akka.actor {
                provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
                deployment {
                  /TheProcess {
                    router = round-robin-group
                    routees.paths = ["/user/TheProcess"] # path of routee on each node
                    # nr-of-instances = 3 # max number of total routees
                    cluster {
                        enabled = on
                        allow-local-routees = off
                        use-role = TheProcess
                    }
                  }                            
                }
                debug {
                  receive = on
                  autoreceive = on
                  lifecycle = on
                  event-stream = on
                  unhandled = on
                }
            }
            akka.remote {
                helios.tcp {
                    # transport-class = "Akka.Remote.Transport.Helios.HeliosTcpTransport, Akka.Remote"
                    # applied-adapters = []
                    # transport-protocol = tcp
                              # public-hostname = "localhost"
                    # 0 or 46001-46010
                    port = 0
                    hostname = "localhost"
                }
                log-remote-lifecyclo-events = DEBUG
            }
            akka.cluster {
              seed-nodes = [
                "akka.tcp://ActorSystem@localhost:2551",
                "akka.tcp://ActorSystem@localhost:2552"
              ]
              roles = [TheSend]
              # auto-down-unreachable-after = 10s
              # how often should the node send out gossip information?
              # gossip-interval = 1s
              # discard incoming gossip messages if not handled within this duration
              # gossip-time-to-live = 2s              
            }
            # http://getakka.net/docs/persistence/at-least-once-delivery
            akka.persistence.at-least-once-delivery.redeliver-interval = 300s
            # akka.persistence.at-least-once-delivery.redelivery-burst-limit =
            # akka.persistence.at-least-once-delivery.warn-after-number-of-unconfirmed-attempts =
            akka.persistence.at-least-once-delivery.max-unconfirmed-messages = 1000000
            akka.persistence.journal.plugin = "akka.persistence.journal.sql-server"
            akka.persistence.journal.publish-plugin-commands = on
            akka.persistence.journal.sql-server {
                class = "Akka.Persistence.SqlServer.Journal.SqlServerJournal, Akka.Persistence.SqlServer"
                plugin-dispatcher = "akka.actor.default-dispatcher"
                table-name = EventJournal
                schema-name = dbo
                auto-initialize = on
                connection-string-name = "AkkaPersistence"
                refresh-interval = 1s
                connection-timeout = 30s
                timestamp-provider = "Akka.Persistence.Sql.Common.Journal.DefaultTimestampProvider, Akka.Persistence.Sql.Common"
                metadata-table-name = Metadata
            }
            akka.persistence.snapshot-store.plugin = ""akka.persistence.snapshot-store.sql-server""
            akka.persistence.snapshot-store.sql-server {
              class = "Akka.Persistence.SqlServer.Snapshot.SqlServerSnapshotStore, Akka.Persistence.SqlServer"
              plugin-dispatcher = ""akka.actor.default-dispatcher""
              connection-string-name = "AkkaPersistence"
              schema-name = dbo
              table-name = SnapshotStore
              auto-initialize = on
            }
      ]]>
  </hocon>

And here is my backend config:

  <hocon>
    <![CDATA[
        akka.loglevel = INFO
        akka.log-config-on-start = on
        akka.stdout-loglevel = INFO
        akka.actor {
            provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
            debug {
              receive = on
              autoreceive = on
              lifecycle = on
              event-stream = on
              unhandled = on
            }
        }
        akka.remote {
          helios.tcp {
                # transport-class = "Akka.Remote.Transport.Helios.HeliosTcpTransport, Akka.Remote"
                # applied-adapters = []
                # transport-protocol = tcp
                        # public-hostname = "localhost"
                # 
                # seed-node ports 2551 and 2552
                # non-seed-node port 0 or 46001-46010
                port = 2551
                hostname = "localhost"
            }
            log-remote-lifecyclo-events = INFO
        }
        akka.cluster {
          seed-nodes = [
            "akka.tcp://ActorSystem@localhost:2551",
            "akka.tcp://ActorSystem@localhost:2552"
          ]
          roles = [TheProcess]
          # auto-down-unreachable-after = 10s
        }
      ]]>
  </hocon>

I think the issue is akka persistence related, what can the issue be?

Finally solved, the persistenceId must be set per IIS instance as stated here:

Identifiers

A persistent actor must have an identifier that doesn't change across different actor incarnations. The identifier must be defined with the PersistenceId method.

so I have put a config key in order to set the correct persistenceId for each specific IIS instance.

There is also a missing else if on the ReceiveCommand that execute a UnstashAll, that cause the multiple deliver.