[Pyrex] possible bug with public function declarations

Robby Dermody robbyd at u20.org
Mon Feb 21 07:05:22 CET 2005


Greg and all,

One more point I might make in support of a feature to support direct 
calls between top level C functions in different modules is that, a 
major selling point of pyrex is that it allows one to write code with 
the speed of C in a simplified manner. This code may then interact with 
python code pretty transparently.

Sometimes, code may be written which does not fit the 'one extension 
type in one file' model commonly touted. For instance, I implemented an 
entire data processing subsystem of my program in pyrex, and it adds up 
to around 14,000 lines of pyrex code, split across about 20 or so files. 
I chose pyrex because while the rest of the program was in python, this 
subsystem must process mass quantities of data as quickly as possible, 
and python would be unrealistic for that, while straight Python/C API 
would be a chore (who wants to count references anyway). The python 
front-end interface to this subsystem is quite minimal -- the system 
takes in data from a variety of sources (e.g. sockets, sniffer 
interface, etc) and does most of the processing internally. While I try 
to keep my design as coherent and modular as possible, C calls across 
file boundaries must be made from time to time, and they are oftentimes 
inline of a time-intensive part of data processing. To me, it would seem 
that the speed benefits of pyrex begin to evaporate once I start 
insulating each file's functionality in an extension type, and then make 
cross-file calls through these extension type objects, as I end up 
incurring all of that pythonic function call overhead when I don't 
really *need* to.

Maybe I just got carried away with pyrex or didn't understand it's 
intended purpose :), but I would sure love some support of this feature.

Robby



More information about the Pyrex mailing list