I need a queue which multiple threads can put stuff into, and multiple threads may read from.
Python has at least two queue classes, queue.Queue
and collections.deque
, with the former seemingly using the latter internally. Both claim to be thread-safe in the documentation.
However, the Queue
docs also state:
collections.deque
is an alternative implementation of unbounded queues with fast atomicappend()
andpopleft()
operations that do not require locking and also support indexing.
Which I guess I don't quite understand: Does this mean deque
isn't fully thread-safe after all?
If it is, I may not fully understand the difference between the two classes. I can see that Queue
adds blocking functionality. On the other hand, it loses some deque
features like support for the in
operator.
Is accessing the internal deque
object directly
x in Queue().queue
thread-safe?
Also, why does Queue
employ a mutex for its operations when deque
is thread-safe already?
queue.Queue
and collections.deque
serve different purposes. queue.Queue
is intended for allowing different threads to communicate using queued messages/data, whereas collections.deque
is simply intended as a data structure. That's why queue.Queue
has methods like put_nowait()
, get_nowait()
, and join()
, whereas collections.deque
doesn't. queue.Queue
isn't intended to be used as a collection, which is why it lacks the likes of the in
operator.
It boils down to this: if you have multiple threads and you want them to be able to communicate without the need for locks, you're looking for queue.Queue
; if you just want a queue or a double-ended queue as a datastructure, use collections.deque
.
Finally, accessing and manipulating the internal deque of a queue.Queue
is playing with fire - you really don't want to be doing that.