I am using the pynetdicom library to receive and process medical dicom images. The processing is performed in the callback function "on_association_released". However, when receiving certain studies, it will cause Python to crash due to what appears to be a child thread crashing.
From the OSX crash report it seems to be libdispatch library that is the cause but not sure how or why.
This is the function:
def on_association_released(self):
if not self.auto_process:
self.incoming = []
return
dicoms = [Dicom(f=x) for x in self.incoming]
self.incoming = []
incoming = Study(dicom_list=dicoms)
log.info("Incoming study: {incoming}".format(**locals()))
completed_tasks = {}
time.sleep(1)
for task in AVAILABLE_PROCESS_TASKS:
log.info("Trying task: {task}".format(**locals()))
process_task = task(study=incoming)
try:
if process_task.valid:
log.info("{incoming} is valid for {process_task}".format(**locals()))
try:
process_task.process()
except Exception as e:
log.warning(
'Failed to perform {process_task} on {incoming}: \n {e}'.format(**locals())
)
else:
log.info("Completed {process_task} for {incoming} !".format(**locals()))
else:
log.warning("{incoming} is not a valid study for {process_task}".format(**locals()))
except Exception as e:
log.warning("{incoming} could not be assessed by {process_task}".format(**locals()))
myemail.nhs_mail(recipients=[admin],
subject=f"dicomserver {VERSION}: Failed to start listener",
message=f"{incoming} could not be assessed by {process_task}: {e.args}"
)
This is the final log message from the application log:
2019-03-15 12:19:06 I [process.py:on_association_released:171] Incoming study: Study(1.2.826.0.1.2112370.55.1.12145941)
This is the OSX Crash Report:
Process: Python [84177]
Path: /Library/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python
Identifier: Python
Version: 3.6.1 (3.6.1)
Code Type: X86-64 (Native)
Parent Process: Python [84175]
Responsible: Terminal [346]
User ID: 503
Date/Time: 2019-03-15 12:19:06.371 +0000
OS Version: Mac OS X 10.11.6 (15G1108)
Report Version: 11
Anonymous UUID: E7340644-9523-1C6B-0B2B-74D6043CFED6
Time Awake Since Boot: 590000 seconds
System Integrity Protection: enabled
Crashed Thread: 1
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000110
VM Regions Near 0x110:
-->
__TEXT 0000000100000000-0000000100001000 [ 4K] r-x/rwx SM=COW /Library/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python
Application Specific Information:
*** multi-threaded process forked ***
crashed on child side of fork pre-exec
This is the top of Thread 1 crash trace:
Thread 1 Crashed:
0 libdispatch.dylib 0x00007fff8e6cc661 _dispatch_queue_push_queue + 345
1 libdispatch.dylib 0x00007fff8e6cab06 _dispatch_queue_wakeup_with_qos_slow + 126
2 libdispatch.dylib 0x00007fff8e6d113f _dispatch_mach_msg_send + 1952
3 libdispatch.dylib 0x00007fff8e6d08dc dispatch_mach_send + 262
4 libxpc.dylib 0x00007fff86858fc9 xpc_connection_send_message_with_reply + 131
5 com.apple.CoreFoundation 0x00007fff8ef43b3f __66-[CFPrefsSearchListSource generationCountFromListOfSources:count:]_block_invoke_2 + 143
6 com.apple.CoreFoundation 0x00007fff8ef4396d _CFPrefsWithDaemonConnection + 381
7 com.apple.CoreFoundation 0x00007fff8ef42af6 __66-[CFPrefsSearchListSource generationCountFromListOfSources:count:]_block_invoke + 150
8 com.apple.CoreFoundation 0x00007fff8ef42893 -[CFPrefsSearchListSource generationCountFromListOfSources:count:] + 179
9 com.apple.CoreFoundation 0x00007fff8ef42174 -[CFPrefsSearchListSource alreadylocked_copyDictionary] + 324
10 com.apple.CoreFoundation 0x00007fff8ef41dbc -[CFPrefsSearchListSource alreadylocked_copyValueForKey:] + 60
11 com.apple.CoreFoundation 0x00007fff8ef41d4c ___CFPreferencesCopyAppValueWithContainer_block_invoke + 60
12 com.apple.CoreFoundation 0x00007fff8ef39a70 +[CFPrefsSearchListSource withSearchListForIdentifier:container:perform:] + 608
13 com.apple.CoreFoundation 0x00007fff8ef397c7 _CFPreferencesCopyAppValueWithContainer + 183
14 com.apple.SystemConfiguration 0x00007fff998b3a9b SCDynamicStoreCopyProxiesWithOptions + 163
15 _scproxy.cpython-36m-darwin.so 0x000000010f0f5a63 get_proxy_settings + 35
16 org.python.python 0x000000010006a604 _PyCFunction_FastCallDict + 436
17 org.python.python 0x00000001000f33e4 call_function + 612
18 org.python.python 0x00000001000f8d84 _PyEval_EvalFrameDefault + 21892
The issue looks awfully similar to a long-standing problem with Python on MacOS.
The root cause as far as I understand it is that fork() is hard to do right if there's threads involved, unless you immediately exec().
MacOS "protects" again the possible pitfalls by crashing a process if it's accessing certain system functionality such as libdispatch if it forked, but didn't exec yet.
Unfortunately these calls can happen in unexpected places, such as in _scproxy.cpython-36m-darwin.so which is shown at position 15 of the stack trace.
There's a number of Python bugs filed about this (1, 2, 3, for example), but there's no silver bullet as far as I know.
In your particular case, it might be possible to prevent the crash by running your Python interpreter with the environment variable no_proxy=*. This should prevent calls to the system configuration framework scproxy to find proxy settings.