I'm trying to get the Py_INCREF's and Py_DECREF's for my c extension right.
Whilst doing so I've been stumbling about really high values for generators. What I've
been doing is the following:
// Forgive me for leaving out the NULL checks
PyObject *get_generator = PyUnicode_InternFromString("get_generator");
PyObject *callback;
PyObject *seq;
PyObject *item;
callback = PyObject_CallMethodObjArgs(parent->state, get_generator, NULL);
seq = PyObject_GetIter(tmp);
Py_DECREF(callback);
while ((item = PyIter_Next(seq))) {
...
Code
...
Py_DECREF(item);
}
Py_DECREF(seq);
printf("!!!%zd\n", seq);
Can somebody explain please.
printf("!!!%zd\n", seq); isn't printing the reference count, it's printing the raw pointer address (slightly incorrectly, since there is no guarantee that a pointer and a signed size_t are compatible for printing).
Also note that it's not safe to check the real reference count if you don't hold a pointer you own (or have borrowed safely), so checking after the Py_DECREF (when you gave up ownership) is undefined behavior.
To fix, change:
Py_DECREF(seq);
printf("!!!%zd\n", seq);
to:
printf("!!!%zd\n", Py_REFCNT(seq));
Py_DECREF(seq);
You should expect to see a count of 1 under normal circumstances, though if the object passed to PyObject_GetIter is itself an iterator (making PyObject_GetIter the identity function, doing nothing but incrementing the reference count and returning the otherwise unmodified argument), then the count could be higher if other references to the same iterator might exist elsewhere.
Assuming they weren't omitted simply for brevity, you do need to add checks for return values; virtually all of your API calls could return NULL if an exception is raised, and silently continuing processing rather than passing the error up immediately risks replacing a (useful) exception message/traceback with an (unhelpful) segfault.