I'm studying the source code of numpy, and I was reading the numpy/_core/multiarray.py file, which contains core functions like concatenate or inner.
These methods are decorated, and the definition of the decorator involves the module _multiarray_umath, which should exist in the file _core, however there is no such file.
Searching in my local installation of numpy, I noticed that in _core there is a file called _multiarray_umath.cpython-313-x86_64-linux-gnu.so, and I guess this is the module actually imported by from . import _multiarray_umath.
What kind of file is _multiarray_umath.cpython-313-x86_64-linux-gnu.so? The extension .so reminds me of shared objects in C++, so I guess this file is compiled during numpy installation, but how is it generated?
The _core folder contains a src/multiarray folder written in C it seems, and I'd like to find there the implementation of concatenate or inner.
Note: there is a module _multiarray_umath in the core folder, but since core is to be removed, I didn't pay much attention to it.
The extension .so reminds me of shared objects in C++, so I guess this file is compiled during numpy installation, but how is it generated?
It is a compiled file based upon ~100 C and C++ sources.
The specifics can be found by building numpy from source, then introspecting the build with meson.
For example, to introspect how all top-level targets were built, you can do the following.
meson introspect --targets -i build/ > targets.json
You can search within this file to find the _multiarray_umath.so target, which provides the following information:
{
"name": "_multiarray_umath.cpython-313-x86_64-linux-gnu",
"id": "7b10d1f@@_multiarray_umath.cpython-313-x86_64-linux-gnu.so@sha",
"type": "shared module",
"defined_in": "/home/user/numpy/numpy/_core/meson.build",
"filename": [
"/home/user/numpy/build/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so"
],
"build_by_default": true,
"target_sources": [
{
"language": "c",
"compiler": [
"ccache",
"cc"
],
"parameters": [
"-I/home/user/numpy/build/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so.p",
"-I/home/user/numpy/build/numpy/_core",
"-I/home/user/numpy/numpy/_core",
"-I/home/user/numpy/build/numpy/_core/include",
"-I/home/user/numpy/numpy/_core/include",
"-I/home/user/numpy/numpy/_core/src/common",
"-I/home/user/numpy/numpy/_core/src/multiarray",
"-I/home/user/numpy/numpy/_core/src/npymath",
"-I/home/user/numpy/numpy/_core/src/umath",
"-I/home/user/numpy/numpy/_core/src/highway",
"-I/home/user/.miniforge3/envs/scipy-dev/include",
"-I/home/user/.miniforge3/envs/scipy-dev/include/python3.13",
"-I/home/user/numpy/build/meson_cpu",
"-fvisibility=hidden",
"-fdiagnostics-color=always",
"-Wall",
"-Winvalid-pch",
"-std=c11",
"-O2",
"-g",
"-fno-strict-aliasing",
"-msse",
"-msse2",
"-msse3",
"-DNPY_HAVE_SSE2",
"-DNPY_HAVE_SSE",
"-DNPY_HAVE_SSE3",
"-march=nocona",
"-mtune=haswell",
"-ftree-vectorize",
"-fPIC",
"-fstack-protector-strong",
"-fno-plt",
"-O2",
"-ffunction-sections",
"-pipe",
"-DNDEBUG",
"-D_FORTIFY_SOURCE=2",
"-O2",
"-fPIC",
"-DHAVE_CBLAS",
"-DNPY_INTERNAL_BUILD",
"-DHAVE_NPY_CONFIG_H",
"-D_FILE_OFFSET_BITS=64",
"-D_LARGEFILE_SOURCE=1",
"-D_LARGEFILE64_SOURCE=1"
],
"sources": [
"/home/user/numpy/numpy/_core/src/multiarray/abstractdtypes.c",
"/home/user/numpy/numpy/_core/src/multiarray/alloc.c",
"/home/user/numpy/numpy/_core/src/multiarray/arrayobject.c",
"/home/user/numpy/numpy/_core/src/multiarray/array_coercion.c",
"/home/user/numpy/numpy/_core/src/multiarray/array_converter.c",
"/home/user/numpy/numpy/_core/src/multiarray/array_method.c",
"/home/user/numpy/numpy/_core/src/multiarray/array_api_standard.c",
"/home/user/numpy/numpy/_core/src/multiarray/array_assign_scalar.c",
"/home/user/numpy/numpy/_core/src/multiarray/array_assign_array.c",
"/home/user/numpy/numpy/_core/src/multiarray/arrayfunction_override.c",
"/home/user/numpy/numpy/_core/src/multiarray/arraywrap.c",
"/home/user/numpy/numpy/_core/src/multiarray/buffer.c",
"/home/user/numpy/numpy/_core/src/multiarray/calculation.c",
"/home/user/numpy/numpy/_core/src/multiarray/compiled_base.c",
"/home/user/numpy/numpy/_core/src/multiarray/common.c",
"/home/user/numpy/numpy/_core/src/multiarray/common_dtype.c",
"/home/user/numpy/numpy/_core/src/multiarray/convert.c",
"/home/user/numpy/numpy/_core/src/multiarray/convert_datatype.c",
"/home/user/numpy/numpy/_core/src/multiarray/conversion_utils.c",
"/home/user/numpy/numpy/_core/src/multiarray/ctors.c",
"/home/user/numpy/numpy/_core/src/multiarray/datetime.c",
"/home/user/numpy/numpy/_core/src/multiarray/datetime_strings.c",
"/home/user/numpy/numpy/_core/src/multiarray/datetime_busday.c",
"/home/user/numpy/numpy/_core/src/multiarray/datetime_busdaycal.c",
"/home/user/numpy/numpy/_core/src/multiarray/descriptor.c",
"/home/user/numpy/numpy/_core/src/multiarray/dlpack.c",
"/home/user/numpy/numpy/_core/src/multiarray/dtypemeta.c",
"/home/user/numpy/numpy/_core/src/multiarray/dragon4.c",
"/home/user/numpy/numpy/_core/src/multiarray/dtype_transfer.c",
"/home/user/numpy/numpy/_core/src/multiarray/dtype_traversal.c",
"/home/user/numpy/numpy/_core/src/multiarray/public_dtype_api.c",
"/home/user/numpy/numpy/_core/src/multiarray/flagsobject.c",
"/home/user/numpy/numpy/_core/src/multiarray/getset.c",
"/home/user/numpy/numpy/_core/src/multiarray/hashdescr.c",
"/home/user/numpy/numpy/_core/src/multiarray/item_selection.c",
"/home/user/numpy/numpy/_core/src/multiarray/iterators.c",
"/home/user/numpy/numpy/_core/src/multiarray/legacy_dtype_implementation.c",
"/home/user/numpy/numpy/_core/src/multiarray/mapping.c",
"/home/user/numpy/numpy/_core/src/multiarray/methods.c",
"/home/user/numpy/numpy/_core/src/multiarray/multiarraymodule.c",
"/home/user/numpy/numpy/_core/src/multiarray/nditer_api.c",
"/home/user/numpy/numpy/_core/src/multiarray/nditer_constr.c",
"/home/user/numpy/numpy/_core/src/multiarray/nditer_pywrap.c",
"/home/user/numpy/numpy/_core/src/multiarray/npy_static_data.c",
"/home/user/numpy/numpy/_core/src/multiarray/number.c",
"/home/user/numpy/numpy/_core/src/multiarray/refcount.c",
"/home/user/numpy/numpy/_core/src/multiarray/sequence.c",
"/home/user/numpy/numpy/_core/src/multiarray/scalarapi.c",
"/home/user/numpy/numpy/_core/src/multiarray/shape.c",
"/home/user/numpy/numpy/_core/src/multiarray/strfuncs.c",
"/home/user/numpy/numpy/_core/src/multiarray/stringdtype/dtype.c",
"/home/user/numpy/numpy/_core/src/multiarray/stringdtype/utf8_utils.c",
"/home/user/numpy/numpy/_core/src/multiarray/stringdtype/static_string.c",
"/home/user/numpy/numpy/_core/src/multiarray/temp_elide.c",
"/home/user/numpy/numpy/_core/src/multiarray/usertypes.c",
"/home/user/numpy/numpy/_core/src/multiarray/vdot.c",
"/home/user/numpy/numpy/_core/src/multiarray/textreading/conversions.c",
"/home/user/numpy/numpy/_core/src/multiarray/textreading/field_types.c",
"/home/user/numpy/numpy/_core/src/multiarray/textreading/growth.c",
"/home/user/numpy/numpy/_core/src/multiarray/textreading/readtext.c",
"/home/user/numpy/numpy/_core/src/multiarray/textreading/rows.c",
"/home/user/numpy/numpy/_core/src/multiarray/textreading/stream_pyobject.c",
"/home/user/numpy/numpy/_core/src/multiarray/textreading/str_to_int.c",
"/home/user/numpy/numpy/_core/src/npymath/arm64_exports.c",
"/home/user/numpy/numpy/_core/src/common/array_assign.c",
"/home/user/numpy/numpy/_core/src/common/gil_utils.c",
"/home/user/numpy/numpy/_core/src/common/mem_overlap.c",
"/home/user/numpy/numpy/_core/src/common/npy_argparse.c",
"/home/user/numpy/numpy/_core/src/common/npy_import.c",
"/home/user/numpy/numpy/_core/src/common/npy_longdouble.c",
"/home/user/numpy/numpy/_core/src/common/ufunc_override.c",
"/home/user/numpy/numpy/_core/src/common/numpyos.c",
"/home/user/numpy/numpy/_core/src/common/npy_cpu_features.c",
"/home/user/numpy/numpy/_core/src/common/npy_cpu_dispatch.c",
"/home/user/numpy/numpy/_core/src/common/blas_utils.c",
"/home/user/numpy/numpy/_core/src/common/cblasfuncs.c",
"/home/user/numpy/numpy/_core/src/common/python_xerbla.c",
"/home/user/numpy/numpy/_core/src/umath/ufunc_type_resolution.c",
"/home/user/numpy/numpy/_core/src/umath/extobj.c",
"/home/user/numpy/numpy/_core/src/umath/legacy_array_method.c",
"/home/user/numpy/numpy/_core/src/umath/override.c",
"/home/user/numpy/numpy/_core/src/umath/reduction.c",
"/home/user/numpy/numpy/_core/src/umath/ufunc_object.c",
"/home/user/numpy/numpy/_core/src/umath/umathmodule.c",
"/home/user/numpy/numpy/_core/src/umath/wrapping_array_method.c",
"/home/user/numpy/numpy/_core/src/umath/_scaled_float_dtype.c"
],
"generated_sources": [
"/home/user/numpy/build/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so.p/arraytypes.c",
"/home/user/numpy/build/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so.p/einsum.c",
"/home/user/numpy/build/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so.p/einsum_sumprod.c",
"/home/user/numpy/build/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so.p/lowlevel_strided_loops.c",
"/home/user/numpy/build/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so.p/nditer_templ.c",
"/home/user/numpy/build/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so.p/scalartypes.c",
"/home/user/numpy/build/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so.p/loops.c",
"/home/user/numpy/build/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so.p/matmul.c",
"/home/user/numpy/build/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so.p/scalarmath.c"
],
"unity_sources": []
},
{
"language": "cpp",
"compiler": [
"ccache",
"c++"
],
"parameters": [
"-I/home/user/numpy/build/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so.p",
"-I/home/user/numpy/build/numpy/_core",
"-I/home/user/numpy/numpy/_core",
"-I/home/user/numpy/build/numpy/_core/include",
"-I/home/user/numpy/numpy/_core/include",
"-I/home/user/numpy/numpy/_core/src/common",
"-I/home/user/numpy/numpy/_core/src/multiarray",
"-I/home/user/numpy/numpy/_core/src/npymath",
"-I/home/user/numpy/numpy/_core/src/umath",
"-I/home/user/numpy/numpy/_core/src/highway",
"-I/home/user/.miniforge3/envs/scipy-dev/include",
"-I/home/user/.miniforge3/envs/scipy-dev/include/python3.13",
"-I/home/user/numpy/build/meson_cpu",
"-fvisibility=hidden",
"-fvisibility-inlines-hidden",
"-fdiagnostics-color=always",
"-D_GLIBCXX_ASSERTIONS=1",
"-Wall",
"-Winvalid-pch",
"-std=c++17",
"-O2",
"-g",
"-msse",
"-msse2",
"-msse3",
"-DNPY_HAVE_SSE2",
"-DNPY_HAVE_SSE",
"-DNPY_HAVE_SSE3",
"-fvisibility-inlines-hidden",
"-fmessage-length=0",
"-march=nocona",
"-mtune=haswell",
"-ftree-vectorize",
"-fPIC",
"-fstack-protector-strong",
"-fno-plt",
"-O2",
"-ffunction-sections",
"-pipe",
"-DNDEBUG",
"-D_FORTIFY_SOURCE=2",
"-O2",
"-fPIC",
"-DHAVE_CBLAS",
"-DNPY_INTERNAL_BUILD",
"-DHAVE_NPY_CONFIG_H",
"-D_FILE_OFFSET_BITS=64",
"-D_LARGEFILE_SOURCE=1",
"-D_LARGEFILE64_SOURCE=1",
"-fno-exceptions",
"-fno-rtti"
],
"sources": [
"/home/user/numpy/numpy/_core/src/multiarray/stringdtype/casts.cpp",
"/home/user/numpy/numpy/_core/src/npysort/quicksort.cpp",
"/home/user/numpy/numpy/_core/src/npysort/mergesort.cpp",
"/home/user/numpy/numpy/_core/src/npysort/timsort.cpp",
"/home/user/numpy/numpy/_core/src/npysort/heapsort.cpp",
"/home/user/numpy/numpy/_core/src/npysort/radixsort.cpp",
"/home/user/numpy/numpy/_core/src/npysort/selection.cpp",
"/home/user/numpy/numpy/_core/src/npysort/binsearch.cpp",
"/home/user/numpy/numpy/_core/src/multiarray/textreading/tokenize.cpp",
"/home/user/numpy/numpy/_core/src/common/npy_hashtable.cpp",
"/home/user/numpy/numpy/_core/src/umath/clip.cpp",
"/home/user/numpy/numpy/_core/src/umath/dispatching.cpp",
"/home/user/numpy/numpy/_core/src/umath/special_integer_comparisons.cpp",
"/home/user/numpy/numpy/_core/src/umath/string_ufuncs.cpp",
"/home/user/numpy/numpy/_core/src/umath/stringdtype_ufuncs.cpp"
],
"generated_sources": [],
"unity_sources": []
},
{
"linker": [
"c++"
],
"parameters": [
"-L/home/user/.miniforge3/envs/scipy-dev/lib",
"-Wl,--as-needed",
"-Wl,--allow-shlib-undefined",
"-shared",
"-fPIC",
"-Wl,-O2",
"-Wl,--sort-common",
"-Wl,--as-needed",
"-Wl,-z,relro",
"-Wl,-z,now",
"-Wl,--disable-new-dtags",
"-Wl,--gc-sections",
"-Wl,--allow-shlib-undefined",
"-Wl,-rpath,/home/user/.miniforge3/envs/scipy-dev/lib",
"-Wl,-rpath-link,/home/user/.miniforge3/envs/scipy-dev/lib",
"-fvisibility-inlines-hidden",
"-fmessage-length=0",
"-march=nocona",
"-mtune=haswell",
"-ftree-vectorize",
"-fPIC",
"-fstack-protector-strong",
"-fno-plt",
"-O2",
"-ffunction-sections",
"-pipe",
"-DNDEBUG",
"-D_FORTIFY_SOURCE=2",
"-O2",
"-Wl,--start-group",
"-Wl,--start-group",
"numpy/_core/libnpymath.a",
"numpy/_core/libunique_hash.a",
"numpy/_core/lib_multiarray_umath_mtargets.a",
"numpy/_core/libhighway.a",
"/home/user/.miniforge3/envs/scipy-dev/lib/libopenblas.so",
"-Wl,--end-group",
"-Wl,--end-group"
]
}
],
"extra_files": [],
"subproject": null,
"dependencies": [
"dep139963354961936",
"openblas",
"python-3.13"
],
"depends": [],
"installed": true,
"install_filename": [
"/usr/lib/python3.13/site-packages/numpy/_core/_multiarray_umath.cpython-313-x86_64-linux-gnu.so"
]
},
Some of the .c files which are included in this .so file are generated C files. You can see this in the generated_sources section. In most cases, these files were generated from a .c.src file. (meson introspect doesn't appear to show how these generated sources were created. I'm not sure why.)
An alternative way to understand how _multiarray_umath.so was built is to read the source code. In NumPy's meson configuration, the way that this module is compiled is described here.