[Pyrex] calling cdef'ed functions between pyrex modules

[Robby Dermody]

Greg,

I got it working, actually. You're right, it isn't supported (it's kind
of a hack...but not too bad of one :).

Say I have 3 files, a.pyx, b.pyx, and c.pyx. What I did was:

1. in my setup.py, have a construct like:
    Extension("a",
        ["a.pyx", "b.pyx", "c.pyx"], 
	....
     )
thus, the a module will be linked with b and c as well. (note that in my
case, b and c have no python interface (i.e. no extension types, python
classes defined...they are just a bunch of C functions at the module
level.

2. say I have a function in b.pyx that needs to be called in a.pyx. I
would define the function as public in b.pyx (as in cdef public void
foo()), and in a.pyx I would have an 'include "b.pxi"' line ...pretty
standard. As the function is now not static in the generated b.c output
file pyrex produces, and the resultant object files from b.c and a.c 
are linked together, a.pyx can now call that function in b.pyx with no
problems. And, it's fast too, no python code is generated around it at
all.
HOWEVER, if I stopped here, if that function in b.pyx that I called from
a.pyx had any python code whatsoever, it would generate a segfault, as
b's __pyx_m variable wasn't initialized (since the pyrex-generated
initb() wasn't ever called).

3. to fix this, all I did was call initb() and initc() in a place in
a.pyx where they would always run. I put them in the __init__ method of
my extension type, as it's the only interface into this all and will
always be called before anything in b.pyx and c.pyx is called. I had to
put a "cdef extern void initb()" in a.pyx as well, of course, to call
initb() from a.pyx.

Informal tests I've done on this show that this works as desired. But I
might be overlooking something....pretty simple overall. I tried putting
the public interfaces of b.pyx and c.pyx in an extension type at first
(and making them as separate extensions in distutils) but that was a
dead end for me.

Okay, I realize it's pretty confusing as I stated it, but maybe this
will save time for someone who wants to do the same kind of thing...

Robby

On Mon, 2004-12-20 at 20:09, Greg Ewing wrote:
> Robby Dermody wrote:
> > For any given one of these pyrex modules, it needs to
> > make calls to cdef functions in any of the other pyrex modules.
> 
> Exporting top-level C functions to another Pyrex module
> isn't currently supported.
> 
> > I really don't
> > want to stick everything in a single .pyx file...it would be a few
> > thousand lines of pyrex code in a single file...
> 
> You could keep the parts in .pxi files included by the
> main .pyx file. There wouldn't be much difference then,
> except for compilation time.
> 
> > I'm guessing that I'll have to encapsulate this C 
>  > functionality as a pyrex extension type
> 
> That's probably the easiest way, if you really want
> them to be separate modules from Python's point of
> view.
> 
>  > But what about the performance of this solution?
> 
> It should be pretty fast. A C slot in the extension
> type instance points to a table of C function pointers.
> If you keep a reference to the instance in a C variable,
> it will only be a couple of memory accesses slower than
> calling a C function directly.
> 
> --
> Greg
> 
> 
> _______________________________________________
> Pyrex mailing list
> Pyrex at lists.copyleft.no
> http://lists.copyleft.no/mailman/listinfo/pyrex
>