[Pyrex] Callbacks from threads and PyGILState_Ensure/PyGILState_Release

[Greg Ewing]

Stefan Behnel wrote:
> There are two cases here. One is a function/method with Python arguments, in
> which case you always need the GIL. But since you can't really call that
> function without already holding the GIL, you will never use "with GIL" in
> this case.

The intention is that "with GIL" would always be the default,
and you would only mark functions as "without GIL" in rare
cases where there was a specific reason.

It might be possible for Pyrex to infer that a plain C function
with no Python arguments or internal operations could be
called without the GIL, but I would have to think carefully
about the implications of that, as it's a form of type
inference. The easiest way is to require an explicit
declaration.

> the
> only way to allow an implementation with Python interaction in subclasses or
> in a specific callback would be to add "with GIL" in the signature, thus
> requiring the GIL even for implementations that do not need it.

That's not strictly true, since you could wrap a "with GIL:"
block around the contents, assuming the existence of such a
feature, or Pyrex could infer the necessity and provide it
automatically if that were considered desirable.

> But it's implementation specific if it can work with or without the GIL

If a particular library calls its callback functions with
the GIL released, then any function you plug into it *must*
be capable of being called without the GIL.

Whether the function actually acquires the GIL or not
is part of its implementation -- if it doesn't touch any
Python data, then it doesn't need to. But the fact that
it can be called without the GIL *is* part of the
signature, whether it's checked by the compiler or not.

There might be a bit of confusion about the meaning of
"without GIL" here. It doesn't mean "this function
acquires the GIL". It just means "this function can be
called without the GIL".

--
Greg