[Pyrex] Problems with 'public' C functions in an extension

[Stefan Behnel]

Hi Greg,

Greg Ewing wrote:
> Stefan Behnel wrote:
> 
>> Maybe the right solution is to drop the current support completely and
>> make
>> public C functions generate a struct with all public function pointers
>> and the
>> above header code for external modules.
> 
> Yes, I've been thinking of doing something like that to
> make it possible to cimport C functions in another Pyrex
> module, but C code should be able to use it as well,
> given suitable declarations.

The header procedure shown on the page I referenced should work for both Pyrex
and C code.


> BTW, I never really envisaged the 'public' mechanism
> being used across dynamically loaded modules, only for
> C code statically linked with the module. I wouldn't
> be surprised if you just can't do that sort of linking
> on some platforms.

As the page states, you can't on some platforms (and therefore shouldn't
anyway). That's why you have to go through Python to pass function pointers.


> On another tack, do you really need to write the code
> which is going to use these functions in C rather than
> Pyrex? Consider that you can use C methods of an extension
> type from another Pyrex module using the existing
> cimport mechanism, and that almost any code you could
> write in C you could also write in Pyrex.

My use case is exactly the one described on the web page: I have an internal
API in the module and need to partially open it up to allow external modules
to call my internal functions. Those modules may be implemented in C or Pyrex,
I don't (want to) care. It's not only about calling extension types and it's
not the public Python API either. My functions use C pointers, so Python can't
access them. One way to do it would be wrapper functions that take PyCObjects,
but they would show up in the Python dir() of the module, although not part of
its Python level API. So the /right/ way to do it would be public functions
that Pyrex generates C-API code for, as described in the page.

I envision the following functionality in Pyrex:
* Pyrex modules declare some functions "public" (or maybe "api")
* the Pyrex compiler generates a static struct {{"name", function}, ...},
  creates a PyCObject for it in initmodule() and stores it in a module
  attribute "__C_API"
* Pyrex also writes a static function pointer declaration for each function
  into the generated header file, as well as a setup function
  "import_module()" that assigns the function pointers from the corresponding
  struct entry.

* external modules include the header file and call the setup function /once/
  in their own initmodule() function before they call the imported functions

This works for both C and Pyrex code.

An alternative would be to move all setup code into the module and provide a
Python level function "init_C_API(c_obj)" with c_obj being a PyCObject that
points to a struct {{"name", &function}, ...}. I guess that's even nicer, as
it avoids having to touch functions that are not used in the external module.
The header would still contain all declarations.

Stefan