[Pyrex] Callbacks from threads and PyGILState_Ensure/PyGILState_Release

[Stefan Behnel]

Greg Ewing wrote:
> Stefan Behnel wrote:
>> Greg Ewing wrote:
>> > Then two
>> > function types would only be considered compatible
>> > if they had the same value for this option.
>>
>> The patch I submitted for Cython has not adopted this behaviour. The
>> reasoning
>> is that two functions can have an identical signature, while one of them
>> requires the GIL in its implementation and the other one does not
>> (maybe it's
>> implemented in C-ish code or something). Still, both can be completely
>> interchangeable, so their signature should not differ.
> 
> The point is, though, that they're *not* interchangeable.
> One of them requires that the GIL be held when it is
> called, and the other doesn't. If you tried to call the
> GIL-requiring function when the GIL wasn't held, it
> would crash.

There are two cases here. One is a function/method with Python arguments, in
which case you always need the GIL. But since you can't really call that
function without already holding the GIL, you will never use "with GIL" in
this case.

The other case, however, is a function/method with plain C arguments which can
be implemented either way: with or without Python interaction. Here, always
requiring the GIL regardless of the implementation would be wrong, but the
only way to allow an implementation with Python interaction in subclasses or
in a specific callback would be to add "with GIL" in the signature, thus
requiring the GIL even for implementations that do not need it. I can't see
why we should enforce that.


> The ultimate goal I have in mind is for Pyrex to keep
> track of whether the GIL is held or not at each point
> in the code, and generate code to acquire/release it
> as appropriate.

You cannot do that for callbacks, which are amongst the most important use
cases for functions that acquire the GIL.


>> One of the implications of having different signatures is that a "with
>> GIL" function would not be assignable to a C function slot
> 
> You would have to declare the function slot with the
> appropriate GIL signature if it were to be capable of
> pointing to a function that can work without the GIL.

But it's implementation specific if it can work with or without the GIL,
that's my point. It's not specific to the signature or the function slot.

Stefan