Work was quiet today, and so the team was directed to do some "self-development". I decided to have some fun calling C functions from Python. I had already had a good time using Rust to speed up Python, but I kept hitting a brick wall whenever I wanted to work with integers greater than the u128 type could hold. I thought that, by using C's famous GMP library, I could overcome this.
So far, I've managed to build a minimal C program which runs, which seems to do what I want, and which - to my eyes - doesn't have anything obviously wrong with it. This is my code:
#include <stdio.h>
#include <gmp.h>
#define BASE 10
void _factorial(int n, mpz_t result) {
int factor;
mpz_t factor_mpz;
mpz_init(result);
mpz_init_set_ui(result, 1);
for (factor = 1; factor <= n; factor++) {
mpz_init(factor_mpz);
mpz_init_set_ui(factor_mpz, factor);
mpz_mul(result, result, factor_mpz);
mpz_clear(factor_mpz);
}
}
char *factorial(int n) {
char *result;
mpz_t result_mpz;
_factorial(n, result_mpz);
mpz_get_str(result, BASE, result_mpz);
mpz_clear(result_mpz);
return result;
}
int main(void) { // This runs without any apparent issues.
char *result = factorial(100);
printf("%s\n", result);
return 0;
}
I then try to call this from Python like so:
from ctypes import CDLL, c_void_p, c_char_p, c_int32, cast
CLIB = CDLL("./shared.so")
cfunc = CLIB.factorial
cfunc.argtypes = [c_int32]
cfunc.restype = c_char_p
raw_pointer = cfunc(100)
result = raw_pointer.decode()
print(result)
I compiled the C code to an .so file using the following command:
gcc main.c -lgmp -fpic -shared -o shared.so
I then ran the above Python script, but unfortunately ran into two issues:
print() statements and prints the correct result, it then hits a segmentation fault.Does anyone know how I can overcome the segmentation fault, and, if there is indeed a memory leak, how I can plug it?
- Although it reaches the
print()statements and prints the correct result, it then hits a segmentation fault.
Your factorial function uses mpz_get_str() incorrectly. Consider:
char *factorial(int n) { char *result; mpz_t result_mpz; _factorial(n, result_mpz); mpz_get_str(result, BASE, result_mpz); mpz_clear(result_mpz); return result; }
mpz_get_str() offers you two alternatives for how the result is to be stored:
It can write the result into large-enough space that the caller specifies. This is triggered by passing a non-null pointer (to the destination space) as the first argument.
It can dynamically allocate sufficient space for the result. This is triggered by passing a null pointer as the first argument.
Either way, it returns a pointer to the first byte of the result.
You are passing a wild pointer as the first argument, with undefined behavior resulting. From your description of the behavior, you are getting some arbitrary program data overwritten by the digit-string output, such that the program successfully prints that, but soon fails because some essential data has been corrupted.
Your best bet would probably be to let GMP allocate the space for you:
char *factorial(int n) {
char *result;
mpz_t result_mpz;
_factorial(n, result_mpz);
result = mpz_get_str(NULL, BASE, result_mpz); // <=== notice this
mpz_clear(result_mpz);
return result;
}
- I'm worried that, in passing an arbitrary-length string from C to Python, there may be some memory leaks.
C does not have arbitrary-length strings. It does have dynamically allocated objects, which can be character arrays whose contents are C strings. Avoiding memory leaks is an exercise in ensuring that dynamically allocated objects are also freed.
If you proceed as I describe above, then you do have to arrange for freeing the memory to which the return value points. I'm uncertain whether Python's ctypes has a specific provision for that, but at minimum, you should be able to use ctypes to pass the pointer obtained from factorial() to the standard library's free() function.