Pipeline, watch() and multi() in redis. How do they really work?

I'm trying to understand the correct use of commands multi and watch for the access to a database Redis.

The context

I'm using:

• the Python Client for Redis redis-py version 3.5.3.
• the version of the Redis server is Redis server v=5.0.5.

Other links

I have made many researches on Internet and I have found some useful link about the main topic of my question:

• this post is useful but not exhaustive
• this post explains well how works the multi command

Example code

I have written and executed the following code where is used the instruction watch associated to an example Redis key called keyWatch:

r = redis.Redis()

def key_incr():
    print('keyWatch before incr = ' + r.get('keyWatch').decode("utf-8"))
    pipe = r.pipeline()
    pipe.watch('keyWatch')
    pipe.multi()
    pipe.incr('keyWatch')
    pipe.execute()
    print('keyWatch after incr = ' + r.get('keyWatch').decode("utf-8"))

key_incr()

Previous code can be correctly executed and if the initial value of keyWatch is 9, the output of the execution is :

keyWatch before incr = 9
keyWatch after incr = 10

If I remove the instruction multi() from the code it becomes:

r = redis.Redis()

def key_incr():
    print('keyWatch before incr = ' + r.get('keyWatch').decode("utf-8"))
    pipe = r.pipeline()
    pipe.watch('keyWatch')
    # NOTE: here the multi() instruction is commented
    #pipe.multi()
    pipe.incr('keyWatch')
    pipe.execute()
    print('keyWatch after incr = ' + r.get('keyWatch').decode("utf-8"))

key_incr()

Its execution raise the following exception:

raise WatchError("Watched variable changed.")
redis.exceptions.WatchError: Watched variable changed.

My need is to avoid that other clients modify the key keyWatch while are executed instructions contained inside the transaction.

The question

Why in my example code the WatchError exception is raised only if the multi() instruction is not present?

Thanks

---

Edit

Use of MONITOR

I have edited my question and I have integrated it by the use of the Redis command monitor.

By redis-cli monitor (MONITOR in the rest of the post) I can see all the requests to the server during the execution of the previous 2 snippets of code.

Monitor info with multi instruction present

For the case where the multi() instruction is present requests are the following:

> redis-cli monitor
OK
1681733993.273545 [0 127.0.0.1:46342] "GET" "keyWatch"
1681733993.273790 [0 127.0.0.1:46342] "WATCH" "keyWatch"
1681733993.273934 [0 127.0.0.1:46342] "MULTI"
1681733993.273945 [0 127.0.0.1:46342] "INCRBY" "keyWatch" "1"
1681733993.273950 [0 127.0.0.1:46342] "EXEC"
1681733993.274279 [0 127.0.0.1:46342] "GET" "keyWatch"  <--- NOTE THE PRESENCE OF THIS GET

Monitor info without multi instruction (WatchError)

For the case without the multi() instruction, MONITOR shows the following requests:

> redis-cli monitor
OK
1681737498.462228 [0 127.0.0.1:46368] "GET" "keyWatch"
1681737498.462500 [0 127.0.0.1:46368] "WATCH" "keyWatch"
1681737498.462663 [0 127.0.0.1:46368] "INCRBY" "keyWatch" "1"
1681737498.463072 [0 127.0.0.1:46368] "MULTI"
1681737498.463081 [0 127.0.0.1:46368] "EXEC"

Also in this second case is present the MULTI instruction, but between it and the EXEC there aren't any requests.
The keyWatch exception is raised by the EXEC instruction in fact the MONITOR does not show the last "GET" "keyWatch" request (look at the first MONITOR log and you find the last "GET" "keyWatch" request).

All this suggest me that the exception is caused by the execution of:
"INCRBY" "keyWatch" "1" outside of the block MULTI/EXEC.

If someone can confirm this and explain better the behavior is appreciated.

Thanks

The WATCH, MULTI, and EXEC are designed to work together. Specifically, calls to MULTI and EXEC allow the queued commands to be executed in isolation. Redis calls this a transaction.

Here's how it works:

MULTI                          <- start queueing commands
INCR someKey                   <- queue this command
SET someOtherKey someValue     <- queue this command
UNLINK someThirdKey            <- queue this command
EXEC                           <- execute all the queued commands

While these commands are being queued, other commands can come in. These other commands might change keys that are part of the transaction, which could be bad. Enter WATCH.

WATCH is used to, well, watch those keys. If they have been changed by the time EXEC is called, EXEC will return an error. You then need to run code to retry the transaction (or maybe generate an error, depending on you needs).

If they haven't been changed, then EXEC executes all the queued commands and life goes on.

It works like this:

WATCH someKey someOtherKey     <- watch these for changes
MULTI                          <- start queueing commands
INCR someKey                   <- queue this command
SET someOtherKey someValue     <- queue this command
UNLINK someThirdKey            <- queue this command
EXEC                           <- either error or execute queued commands

Note that you don't have to watch a key as part of a transaction, as my example above shows. In this case, I don't care if someone changes the thing I'm going to delete.

Transaction in Redis aren't what developers often think they are. If you want to get deeper in the details, there is a guide to transactions on the Redis website.