I am writing multi-threaded python application.
The main thread create a Thread pool of 5 Worker Threads. The main thread also create a Monitor Thread.
Total: 6 threads + 1 Main Thread = 7
All Threads talk with a MySQL server (mysqldb -> libmysqlclient_r)
In my SQL wrapper i have added a Threading.Lock to the DB query function. This Lock is a Global lock and all the Threads that query the DB, uses it.
def query(self, query):
with lock:
execute Query Here
Everything works well, until at some point, the main thread stuck (and so does all the threads) I attached the GDB debugger and noticed that: (info threads)
7 Thread 0x7f555c386700 (LWP 16077) 0x00007f5561a503c0 in sem_wait () from /lib/libpthread.so.0
6 Thread 0x7f555bb85700 (LWP 16078) 0x00007f5561a503c0 in sem_wait () from /lib/libpthread.so.0
5 Thread 0x7f555b384700 (LWP 16079) 0x00007f5561a503c0 in sem_wait () from /lib/libpthread.so.0
4 Thread 0x7f555ab83700 (LWP 16080) 0x00007f5561a503c0 in sem_wait () from /lib/libpthread.so.0
3 Thread 0x7f555a382700 (LWP 16081) 0x00007f5561a503c0 in sem_wait () from /lib/libpthread.so.0
2 Thread 0x7f5559b81700 (LWP 16083) 0x00007f55609141a3 in select () from /lib/libc.so.6
1 Thread 0x7f5561e6f700 (LWP 16061) 0x00007f5561a503c0 in sem_wait () from /lib/libpthread.so.0
Th1 = Main Thread, Th2 = Monitor Thread, Th3-Th7 - Worker Threads
I noticed that All the threads beside the monitor thread are waiting on sem_wait():
(gdb) bt
#0 0x00007f5561a503c0 in **sem_wait** () from /lib/libpthread.so.0
#1 0x00000000004d44e8 in **PyThread_acquire_lock** ()
#2 0x00000000004d8982 in ?? ()
#3 0x00000000004a7ba5 in PyEval_EvalFrameEx ()
However, the monitor thread is capable of acquiring and releasing the lock (it runs every 30 sec, the select() you see is due to the sleep(30)). I do not understand , why the rest of the threads are stuck on sem_wait(), since no one acquired the lock.
Any ideas how to solve this? how to debug this?
Thank you
It seems like i was using the same connection using multiple threads, which by the DOCS is not allowed.
Tip: Make sure that each Thread has it's own Connection object.