[Pyrex] Callbacks from other threads (crash in PyGILState_Release)

[Geoff Schmidt]

Jani,

>> I tried adding the extra PyEval_Save/RestoreThread that you show in
>> your example, but I still get the same segfault. The revised code is
>> below. Could you take a look and tell me if I understood your advise?
>>
> I think you need to call PyEval_InitThreads() before pthread_create.

Thank you so much. This solved the problem.

For anyone finding this thread with Google, here is the resulting  
test program, showing the correct way to asynchronously call a Python  
callback from another, different C thread in Pyrex:

--- foo.pyx ---
cdef extern from "stdio.h":
     int printf(char *str, ...)

cdef extern from "Python.h":
     ctypedef int PyGILState_STATE
     PyGILState_STATE PyGILState_Ensure()
     void PyGILState_Release(PyGILState_STATE gstate)
     void PyEval_InitThreads()

cdef extern from "pthread.h":
     ctypedef void *pthread_t # it'll do
     int pthread_create(pthread_t *thread, void *attr,
                        void *(*start_routine)(void *), void *arg)

cdef extern void *func(void *x):
     printf("Entering func(%p)\n", x)

     cdef PyGILState_STATE st
     printf("PyGILState_Ensure\n")
     st = PyGILState_Ensure()
     printf("PyGILState_Release\n")
     PyGILState_Release(st)
     printf("Leaving func\n")

def callFuncDirectly():
     func(NULL)

def callFuncInThread():
     PyEval_InitThreads()

     cdef pthread_t thr
     pthread_create(&thr, NULL, func, NULL);

--- setup.py ---

from distutils.core import setup
from distutils.extension import Extension
from Pyrex.Distutils import build_ext

setup(
     name = 'foo',
     ext_modules = [Extension("foo", ["foo.pyx"])],
     cmdclass = {'build_ext': build_ext},
     )

--- end ---

Example usage:

$ python setup.py build_ext --inplace
$ python
 >>> import foo
 >>> foo.callFuncInThread

Geoff Schmidt
gschmidt at gschmidt.org