How to configure mosquitto to retain sessions? Why is it discarding my sessions out of the box?

I'm trying to use paho-mqtt (python client) persuade mosquitto to retain a session between client connects, but every time I connect (with the same user,password,client id) the session is not retained.

My mosquitto configuration is trivial:

listener 1883
password_file /mosquitto/config/passwords

Besides creating a password file with the username and password for that one user, there is no other config except for what may ship with the docker image.

I've traced through the MQTT session at the byte level and read read the values based on the official MQTT5 standard.

I've confirmed that:

The client is connecting with protocol MQTT 5
The client CONNECT messsage includes the correct:
- username
- password
- client id abcde
- clean session flag (0)
- keep alive value (120)
The server CONNACK message has:
- No assigned client ID (wasn't expecting one)
- Session present 0
I can use the python/paho connection to talk to other clients through Mosquitto
I've also tried doing this and killing the connection without letting any DISCONNECT packet be sent.

I don't think this is relavent, but here's how I connect to Mosquitto:

import paho.mqtt.client as mqtt

    mqttc = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, protocol=mqtt.MQTTv5, client_id="abcdef")
    mqttc.on_connect = on_connect
    mqttc.on_message = on_message
    mqttc.username_pw_set("guest", "guest")
    mqttc.connect("localhost", 1884, clean_start=False, keepalive=120)

I've repeatedly connected with the same client Id and created a subscription each time.

Why would Mosquitto not retain sessions out of the box. Am I missing a configuration option?

Solution

Your issue is most likely due to the way you are connecting, rather than an issue with the configuration of Mosquitto (Mosquitto supports persistent sessions, in memory, with it's default configuration).

With V5 there are two relevant settings in the CONNECT packet, Clean Start and Session Expiry Interval. You are setting the first, but not the second and:

If the Session Expiry Interval is absent the value 0 is used. If it is set to 0, or is absent, the Session ends when the Network Connection is closed.

If you alter your code as follows you should be able to establish a session that outlasts the connection:

import paho.mqtt.client as mqtt

def on_connect(mqttc, obj, flags, reason_code, properties):
    print("reason_code: " + str(reason_code))
    print("session_present: " + str(flags.session_present))

mqttc = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, protocol=mqtt.MQTTv5, client_id="abcdef")
mqttc.on_connect = on_connect

properties=mqtt.Properties(mqtt.PacketTypes.CONNECT)
properties.SessionExpiryInterval=240 # seconds
mqttc.connect("localhost", 1883, clean_start=False, properties=properties, keepalive=120)

mqttc.loop_forever()

Running this on my machine I get:

> python test.py
reason_code: Success
session_present: False
KeyboardInterrupt

> python test.py
reason_code: Success
session_present: True

For practical reasons, the session expiry should be longer than keepalive to ensure that the client noticies it has been disconnected before the server discards the session.

Please note that the Paho Python client python does not currently store session info to disk:

This means that when the client is restarted (not just reconnected, the object is recreated usually because the program was restarted) the session is lost. This results in a possible message loss.

Paho Python client also misbehaves when the server discards a session after SessionExpiryInterval resulting in incorrect duplication of QOS2 messages. See Known limitations