OpenSSL: 3.0.8 Python: 3.9.16
I enabled FIPS using below script but FIPS functionality is not working when I use custom libcrypto path but working fine if I use default path. custom path:/tmp/fips_files/libcrypto.so.3 default path: usr/local/ssl/lib/libcrypto.so.3
#!/usr/bin/env python
import ctypes as cts
import sys,hashlib
#PATH_LIBCRYPTO = "/usr/local/ssl/lib/libcrypto.so.3"
PATH_LIBCRYPTO = "/tmp/fips_files/libcrypto.so.3"
POSSL_LIB_CTX = cts.c_void_p
POSSL_PROVIDER = cts.c_void_p
def main(*argv):
libcrypto = cts.CDLL(PATH_LIBCRYPTO)
OSSL_PROVIDER_set_default_search_path = libcrypto.OSSL_PROVIDER_set_default_search_path
OSSL_PROVIDER_set_default_search_path.argtypes = (POSSL_LIB_CTX, cts.c_char_p)
OSSL_PROVIDER_set_default_search_path.restype = cts.c_int
OSSL_LIB_CTX_load_config = libcrypto.OSSL_LIB_CTX_load_config
OSSL_LIB_CTX_load_config.argtypes = (POSSL_LIB_CTX, cts.c_char_p)
OSSL_LIB_CTX_load_config.restype = cts.c_int
OSSL_PROVIDER_load = libcrypto.OSSL_PROVIDER_load
OSSL_PROVIDER_load.argtypes = (POSSL_LIB_CTX, cts.c_char_p)
OSSL_PROVIDER_load.restype = POSSL_PROVIDER
res = OSSL_PROVIDER_set_default_search_path(None, b"/tmp/fips_files/")
if res == 0:
print("OSSL_PROVIDER_set_default_search_path error")
return 1
res = OSSL_LIB_CTX_load_config(None, b"/tmp/fips_files/openssl.cnf")
if res == 0:
print("OSSL_LIB_CTX_load_config error")
return 1
base_provider = OSSL_PROVIDER_load(None, b"base")
fips_provider = OSSL_PROVIDER_load(None, b"fips")
print(base_provider, fips_provider)
libcrypto.EVP_default_properties_is_fips_enabled.argtypes = [cts.c_void_p]
libcrypto.EVP_default_properties_is_fips_enabled.restype = cts.c_int
fips_status = libcrypto.EVP_default_properties_is_fips_enabled(None)
print("fips status before:",fips_status)
libcrypto.EVP_default_properties_enable_fips.argtypes = [cts.c_void_p, cts.c_int]
libcrypto.EVP_default_properties_enable_fips.restype = cts.c_int
ret_status = libcrypto.EVP_default_properties_enable_fips(None, 1)
print ("return :", ret_status)
fips_status = libcrypto.EVP_default_properties_is_fips_enabled(None)
print("fips status after :",fips_status)
print(hashlib.sha1("test_str".encode('utf-8')).hexdigest())
print(hashlib.md5("test_str".encode('utf-8')).hexdigest())
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
sys.exit(rc)
With cutom libcrypto path FIPS not WORKING: PATH_LIBCRYPTO = "/tmp/fips_files/libcrypto.so.3"
root@gssd-ubuntu184:~/Pradeep/FIPS# python new_load_conf_file.py fips base
Python 3.9.16 (main, Feb 20 2023, 12:42:50) [Clang 6.0.0 (tags/RELEASE_600/final)] 064bit on linux
94512782592400 94512782460240
fips status before: 1
return : 1
fips status after : 1
f9a90e7c1ff51236191623b84267d110c617118a
74e710825309d622d0b920390ef03edf
With default libcrypto path FIPS WORKING: PATH_LIBCRYPTO = "/usr/local/ssl/lib/libcrypto.so.3"
root@gssd-ubuntu184:~/Pradeep/FIPS# python new_load_conf_file.py fips base
Python 3.9.16 (main, Feb 20 2023, 12:42:50) [Clang 6.0.0 (tags/RELEASE_600/final)] 064bit on linux
94480037288352 94480037156144
fips status before: 1
return : 1
fips status after : 1
f9a90e7c1ff51236191623b84267d110c617118a
Traceback (most recent call last):
File "/root/Pradeep/FIPS/new_load_conf_file.py", line 63, in <module>
rc = main(*sys.argv[1:])
File "/root/Pradeep/FIPS/new_load_conf_file.py", line 56, in main
print(hashlib.md5("test_str".encode('utf-8')).hexdigest())
ValueError: [digital envelope routines] unsupported
Could you please help me in making FIPS work using libs in the custom path.
Listing [Python.Docs]: hashlib - Secure hashes and message digests.
I want to start with a note: it is possible to have multiple instances of the "same" library loaded in a process, as shown in [SO]: Independent CDLL Library Instances with Ctypes (@CristiFati's answer).
Check [SO]: How to enable FIPS mode for libcrypto and libssl packaged with Python? (@CristiFati's answer).
It's a duplicate of the referenced (closed) question at the end (Update #0 section).
What happens:
Default LibCrypto (if found) is loaded by import hashlib
You load "your own" LibCrypto version, where you set FIPS mode on
HashLib on the other hand, is still using the default instance (where FIPS is not enabled)
I tested with OpenSSL 3.0.8.
Output - Preliminary (note that I'll be reusing this console):
[cfati@cfati-5510-0:/mnt/e/Work/Dev/StackExchange/StackOverflow/q075913071]> ~/sopr.sh ### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ### [064bit prompt]> # Test OpenSSL [064bit prompt]> /usr/local/pc064/openssl/openssl/3.0.8/bin/openssl md5 ./code00.py FATAL: Startup failure (dev note: apps_startup()) for /usr/local/pc064/openssl/openssl/3.0.8/bin/openssl 80E2A959297F0000:error:80000002:system library:process_include:No such file or directory:crypto/conf/conf_def.c:805:calling stat(fipsmodule.cnf) 80E2A959297F0000:error:07800069:common libcrypto routines:provider_conf_load:provider section error:crypto/provider_conf.c:156:section=fips_sect not found 80E2A959297F0000:error:0700006D:configuration file routines:module_run:module initialization error:crypto/conf/conf_mod.c:270:module=providers, value=provider_sect retcode=-1 [064bit prompt]> [064bit prompt]> # Local configuration stuff [064bit prompt]> OPENSSL_CONF_INCLUDE=/usr/local/pc064/openssl/openssl/3.0.8/ssl /usr/local/pc064/openssl/openssl/3.0.8/bin/openssl md5 ./code00.py Error setting digest 80123BCEDC7F0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:373:Global default library context, Algorithm (MD5 : 102), Properties () 80123BCEDC7F0000:error:03000086:digital envelope routines:evp_md_init_internal:initialization error:crypto/evp/digest.c:254:
Check [GitHub]: openssl/openssl - openssl.cnf failed to load fispmodules.cnf if use without full path in 3.0.1 (@CristiFati's comment) for details regarding OPENSSL_CONF_INCLUDE.
Since OpenSSL build (with FIPS support) is valid, moving to Python stuff. Here's an updated version of your code:
code00.py:
#!/usr/bin/env python
import ctypes as cts
import hashlib
import os
import sys
#LIBCRYPTO = "/usr/local/ssl/lib/libcrypto.so.3"
CUSTOM_PATH = "/tmp/fips_files"
LIBCRYPTO = os.path.join(CUSTOM_PATH, "libcrypto.so.3")
CUSTOM_PATH = "/usr/local/pc064/openssl/openssl/3.0.8" # @TODO - cfati: Custom build path
LIBCRYPTO = os.path.join(CUSTOM_PATH, "lib", "libcrypto.so") # @TODO - cfati: Custom build lib path
POSSL_LIB_CTX = cts.c_void_p
POSSL_PROVIDER = cts.c_void_p
def main(*argv):
libcrypto = cts.CDLL(LIBCRYPTO)
OSSL_PROVIDER_set_default_search_path = libcrypto.OSSL_PROVIDER_set_default_search_path
OSSL_PROVIDER_set_default_search_path.argtypes = (POSSL_LIB_CTX, cts.c_char_p)
OSSL_PROVIDER_set_default_search_path.restype = cts.c_int
OSSL_LIB_CTX_load_config = libcrypto.OSSL_LIB_CTX_load_config
OSSL_LIB_CTX_load_config.argtypes = (POSSL_LIB_CTX, cts.c_char_p)
OSSL_LIB_CTX_load_config.restype = cts.c_int
OSSL_PROVIDER_load = libcrypto.OSSL_PROVIDER_load
OSSL_PROVIDER_load.argtypes = (POSSL_LIB_CTX, cts.c_char_p)
OSSL_PROVIDER_load.restype = POSSL_PROVIDER
EVP_default_properties_is_fips_enabled = libcrypto.EVP_default_properties_is_fips_enabled
EVP_default_properties_is_fips_enabled.argtypes = (POSSL_LIB_CTX,)
EVP_default_properties_is_fips_enabled.restype = cts.c_int
EVP_default_properties_enable_fips = libcrypto.EVP_default_properties_enable_fips
EVP_default_properties_enable_fips.argtypes = (POSSL_LIB_CTX, cts.c_int)
EVP_default_properties_enable_fips.restype = cts.c_int
provider_search_path = CUSTOM_PATH.encode()
provider_search_path = os.path.join(CUSTOM_PATH, "lib", "ossl-modules").encode() # @TODO - cfati: Custom mod location
res = OSSL_PROVIDER_set_default_search_path(None, provider_search_path)
if res == 0:
print("OSSL_PROVIDER_set_default_search_path error")
return 1
cfg_search_path = provider_search_path
cfg_search_path = os.path.join(CUSTOM_PATH, "ssl").encode() # @TODO - cfati: Custom cfg location
res = OSSL_LIB_CTX_load_config(None, os.path.join(cfg_search_path, b"openssl.cnf"))
if res == 0:
print("OSSL_LIB_CTX_load_config error")
return 1
base_provider = OSSL_PROVIDER_load(None, b"base")
fips_provider = OSSL_PROVIDER_load(None, b"fips")
print(base_provider, fips_provider)
fips_status = EVP_default_properties_is_fips_enabled(None)
print("EVP_default_properties_is_fips_enabled: {:d}".format(fips_status))
res = EVP_default_properties_enable_fips(None, 1)
print ("EVP_default_properties_enable_fips: {:d}".format(res))
fips_status = EVP_default_properties_is_fips_enabled(None)
print("EVP_default_properties_is_fips_enabled: {:d}".format(fips_status))
print("HL mod: {:}".format(getattr(hashlib, "_hashlib", None)))
print("SHA1: {:s}".format(hashlib.sha1("test_str".encode('utf-8')).hexdigest()))
print("MD5: {:s}".format(hashlib.md5("test_str".encode('utf-8')).hexdigest()))
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
print("\nDone.\n")
sys.exit(rc)
Output:
[064bit prompt]> # Attempt #0. [064bit prompt]> OPENSSL_CONF_INCLUDE=/usr/local/pc064/openssl/openssl/3.0.8/ssl python3.9 ./code00.py Python 3.9.16 (main, Dec 7 2022, 01:11:51) [GCC 9.4.0] 064bit on linux 20919264 20902112 EVP_default_properties_is_fips_enabled: 0 EVP_default_properties_enable_fips: 1 EVP_default_properties_is_fips_enabled: 1 HL mod: <module '_hashlib' from '/usr/lib/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so'> SHA1: f9a90e7c1ff51236191623b84267d110c617118a MD5: 74e710825309d622d0b920390ef03edf Done. [064bit prompt]> [064bit prompt]> # Attempt #1. [064bit prompt]> LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/pc064/openssl/openssl/3.0.8/lib OPENSSL_CONF_INCLUDE=/usr/local/pc064/openssl/openssl/3.0.8/ssl python3.9 ./code00.py Python 3.9.16 (main, Dec 7 2022, 01:11:51) [GCC 9.4.0] 064bit on linux 36180960 36163808 EVP_default_properties_is_fips_enabled: 0 EVP_default_properties_enable_fips: 1 EVP_default_properties_is_fips_enabled: 1 HL mod: <module '_hashlib' from '/usr/lib/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so'> SHA1: f9a90e7c1ff51236191623b84267d110c617118a MD5: 74e710825309d622d0b920390ef03edf Done. [064bit prompt]> [064bit prompt]> ldd /usr/lib/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so linux-vdso.so.1 (0x00007ffe72fe9000) libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007ff3fb0f9000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff3faf07000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff3faf01000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff3faede000) /lib64/ld-linux-x86-64.so.2 (0x00007ff3fb405000) [064bit prompt]> [064bit prompt]> # Attempt #2. [064bit prompt]> ln -s /usr/local/pc064/openssl/openssl/3.0.8/lib/libcrypto.so ./libcrypto.so.1.1 [064bit prompt]> LD_LIBRARY_PATH=.:${LD_LIBRARY_PATH}:/usr/local/pc064/openssl/openssl/3.0.8/lib OPENSSL_CONF_INCLUDE=/usr/local/pc064/openssl/openssl/3.0.8/ssl python3.9 ./code00.py Python 3.9.16 (main, Dec 7 2022, 01:11:51) [GCC 9.4.0] 064bit on linux 43861088 43843936 EVP_default_properties_is_fips_enabled: 0 EVP_default_properties_enable_fips: 1 EVP_default_properties_is_fips_enabled: 1 HL mod: None SHA1: f9a90e7c1ff51236191623b84267d110c617118a MD5: 74e710825309d622d0b920390ef03edf Done.
Since (default) Python's HashLib is built with OpenSSL 1.1.*, and there's a big API / ABI difference between that and v3.*, it doesn't work (as seen, in the 3rd attempt I tried a (lame) workaround (gainarie), but HashLib bindings to OpenSSL failed to load, and default implementations were used).
In order to get past this, you need to build Python with OpenSSL 3.* (not sure if it works OOTB). Check [SO]: How to compile python3 on RHEL with SSL? SSL cannot be imported (@CristiFati's answer) for details (although versions differ).
Anyway, setting LD_LIBRARY_PATH should do the trick. In your case, something like:
LD_LIBRARY_PATH=/tmp/fips_files:${LD_LIBRARY_PATH} python ./code00.py
As I stated in your other question ([SO]: Access OpenSSL FIPS APIs from python ctypes not working (@CristiFati's answer)), things might be easier if using some specialized Python modules (OpenSSL wrappers).