I'm not python
programmer.
But I need to write middleware for Openstack Swift. I don't know what environment the code running in. I have no control over the main thread.
I'm tyring to start sub-thread that will service the queue, but this thread blocks main thread.
The main thread prints 1
and hangs. Pressing Ctrl+C leads to contine execution, but sub-thread stops.
Where to dig?
class ProxyLoggingMiddleware(object):
"""
Middleware that logs Swift proxy requests in the swift log format.
"""
def __init__(self, app, conf, logger=None):
self.queue = Queue(0)
print "1\n"
self.processor = self.init_queue_processor()
print "2\n"
def init_queue_processor(self):
processor = threading.Thread(target=self.process_queue, args=(self.fifo_pipe_pathname, self.queue, self.logger))
processor.setDaemon(True)
processor.start()
return processor
@staticmethod
def process_queue(fifo_pipe_pathname, queue, logger):
json_encoder = json.JSONEncoder()
while True:
stat = queue.get(True) # <----------------------- Blocks here
UPD: I add next code in the middleware and it prints 1, 2, 3
def m():
print "--1\n"
time.sleep(3)
print "--2\n"
t = threading.Thread(target=m)
t.daemon = True
t.start()
print "--3\n"
But should 1 3 2
THE PROBLEM OCCURS ONLY WHEN THIS CODE RUNS IN Openstack Swift environment!
UPD2:
/opt/swift # python --version
Python 2.7.18
/opt/swift # uname -a
Linux 45cefc56fd0a 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 x86_64 Linux
UPD3:
I updated my code
def __init__(self, app, conf, logger=None):
print threading.currentThread()
self.processor = self.init_queue_processor()
def init_queue_processor(self):
processor = threading.Thread(target=self.process_queue, args=(self.fifo_pipe_pathname, self.queue, self.logger))
processor.setDaemon(True)
processor.start()
return processor
@staticmethod
def process_queue(fifo_pipe_pathname, queue, logger):
print threading.currentThread()
json_encoder = json.JSONEncoder()
while True: ....
The output is
/opt/swift # /usr/bin/python /usr/local/bin/swift-proxy-server /etc/swift/proxy-server.conf
<_MainThread(MainThread, started 140613464898888)>
<_MainThread(MainThread, started 140613464898888)>
I see process_queue
executing in main thread.
OMG, I found help in the Swift sources.
Openstack Swift uses green threads from eventlet
.
It need to use another implementation of known classes
from eventlet import sleep
from eventlet.green import threading
instead of the standard python threading, because it patched, how I understod.
so it's explains why the Main
thread using everywhere.